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Important  special  case  of  weak  lumpability  from  a  MRP  to  a  renewal  process.  What  is 
shown  here  is  that  in  an  appropriate  modified  form,  these  concepts  are  important  in 
both  application  and  in  the  foundational  study  of  MRP's. 

Equivalence  and  collapsibility  between  MRP's  are  defined,  and  necessary, 
sufficient,  and  necessary  and  sufficient  conditions  are  given  for  them.  It  is 
shown  that  equivalence,  collapsibility,  weak  lumpability  and  strong  lumpability 
are  morphisms  between  MRP's,  and  their  relations  to  one  another  are  examined. 

Equivalence  between  a  MRP  and  a  renewal  process  is  examined  in  detail.  Specific 
results  are  obtained  for  irreducible,  reducible,  periodic  and  transient  MRP's.  These 
results  are  applied  to  problems  concerning  flows  in  queueing  networks.  It  is  shown 
that  several  well  known  results  in  queueing  theory  are  examples  of  equivalence  (for 
Instance  Burke's  Theorem).  New  and  simpler  proofs  are  given  for  them.  Some  questions, 
previously  unresolved,  are  answered  using  the  techniques  developed  here;  most  notably 
the  question  of  when  the  input  process  to  the  M/M/1  queue  with  instantaneous  Bernoulli 
feedback  is  renewal. 

Convolutions  of  MRP's  are  examined,  and  conditions  are  given  for  equivalence  to  be 
preserved  under  convolution.  It  is  also  shown  that  an  important  class  of  Markov- 
renewal  equations  can  be  simplified  if  the  underlying  MRP  is  equivalent  to  the  renewal 
process. 

Finally,  it  is  shown  that  the  ideas  developed  here  can  be  extended  to  MRP's  on 
general  state  spaces.  The  definitions  of  equivalence,  collapsibility,  weak 
lumpability  and  strong  lumpability  are  given  in  the  general  setting.  Examples  from 
queueing  theory  that  would  make  use  of  the  generalized  results  are  given. 
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CHAPTER  I 
INTRODUCTION 


1.  Introduction 

The  main  purpose  of  this  work  is  to  introduce  some  new  concepts 

in  Markov-renewal  theory  which  are  of  theoretical  and  practical  interest. 

A  Markov-renewal  process  (MRP)  is  a  sequence  of  pairs  of  random 

variables  {X  ,T  }  where  X  takes  values  in  some  measure  space  (S,^e/)  and 
n  n  n 

Tq  takes  values  in  [0,“],  that  satisfies 

P(Vl  '  A-  Tn+1  ±  ‘I  V'-V  V-Tl>  -P(X„M  *  A'  T„«  i  tlX„>  «••• 

for  n«l,2,*»*,  A  e  jed,  t  e  10,®]-  It  is  assumed  that  there  is  a  transi¬ 
tion  function  Q  .(t)  =  P(X  e  A,  T  ..  <  1 1 X  =  x)  which  is  called  the 
kernel  of  the  MRP.  It  is  clear  that  the  kernel  and  the  initial  distribu¬ 
tion  y  (i.e.  y(A)  *  P(Xq  e  A))  completely  specifies  the  distribution  of 
the  process  ^xn»Tn^ 

It  would  seem  that  one  of  the  basic  questions  in  Markov-renewal 
theory  would  be  to  determine  when  two  MRP's  with  different  characteriza¬ 
tions  (i.e.  different  state  space,  kernel,  or  initial  distribution)  are 
"the  same".  A  review  of  the  literature  shows  that  this  question  has 
been  largely  ignored.  Actually,  the  only  work  done  in  this  area  has 
been  the  extension  of  the  concept  of  strong  and  weak  lumpability  between 
Markov  Chains  to  strong  and  weak  lumpability  between  MRP's.  A  descrip¬ 
tion  and  review  of  those  results  will  be  given  in  the  next  section. 

Even  the  present  knowledge  of  strong  and  weak  lumpability 
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between  MRP's  is  quite  incomplete  since  no  attempt  is  ever  made  to  apply 
these  ideas  or  to  see  if  and  when  they  occur  in  practice. 

Consider  the  departure  process  from  an  M/M/l/N  queue  (N  <  °») . 

It  can  he  shown  ((4),  for  Instance)  that  the  departure  process  is  most 
naturally  characterized  as  a  MRP  with  N  +  l  states  (corresponding  to  the 
length  of  the  queue  the  instant  after  a  departure).  When  N  - »  (i.e.  the 
M/M/1  queue),  a  countably  infinite  state  MRP  characterizes  the  departure 
process.  It  is  well  known,  though,  [2],  [4],  that  the  departure  process 
from  a  steady  state  M/M/1  queue  is  a  Poisson  process,  which  like  any 
renewal  process  is  a  one  state  MRP.^  This  is  a  nontrivial  example  of  a 
MRP  with  two  different  characterizations.  Several  other  similar  examples 
can  be  found  (see  section  2.4). 

Attempting  to  construct  definitions  for  two  MRP's  being  "the 
same"  leads  to  the  construction  of  morphias  between  MRP's.  Four 
morphisms  will  be  defined;  strong  lumpability,  weak  lumpability, 
collapslbility  and  equivalence.  Although  strong  and  weak  lumpability 
have  never  been  defined  outside  the  realm  of  finite  state  MRP's,  there 
is  no  problem  extending  their  definitions  to  the  general  case. 

The  conditions  for  strong  lumpability  are  very  restrictive  and 
although  they  can  be  written  down  in  a  mathematically  attractive  form, 
strong  lumpability  is  not  as  interesting  a  morphism  as  the  others.  Weak 
lumpability  is  a  more  interesting  morphism,  but  the  mathematics 

*A  renewal  process  is  a  one  state  MRP  since  there  is  only  one 
kind  of  event.  Sometimes  the  total  number  of  renewals  up  to  time  t 
(the  counting  process)  is  called  the  state  of  the  renewal  process.  In 
that  context  the  state  of  a  renewal  process  can  take  values  in  the 
nonnegative  integers.  In  our  context  though,  the  state  of  a  MRP  at  the 
time  of  an  event  specifies  the  distribution  of  the  time  until  the  next 
event.  Thus,  a  renewal  process  need  have  only  one  state  since  those 
times  are  i.i.d.  random  variables. 
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describing  it  is  not  very  pleasant.  Strong  lumpability  always  implies 
weak  lumpability  but  the  converse  is  not  true. 

From  a  mathematical  point  of  view,  collapsibility  is  the  most 
pleasing  of  the  four  raorphisms.  It  is  easy  to  show  that  weak  lumpability 
implies  collapsibility  but  the  converse  is  presently  unresolved  (due  to 
the  complexity  of  weak  lumpability).  Strong  evidence  is  given  in  sec¬ 
tion  2.6  towards  the  belief  that  collapsibility  is  a  weaker  condition 
than  weak  lumpability. 

The  weakest  of  the  morphisms  is  equivalence.  From  a  practical 
point  of  view,  though,  it  is  the  most  important.  Equivalence  is  exactly 
the  condition  that  allows  one  to  substitute  one  MRP  for  another  in  (say) 
a  queueing  network  without  changing  any  of  the  important  aspects  of  the 
system.  It  is  easy  to  show  that  equivalence  is  weaker  than  the  other 
three  morphisms.  It  is  also  easy  to  show  that  in  the  important  special 
case  where  one  of  the  MRP's  is  a  renewal  process,  weak  lumpability 
collapsibility  and  equivalence  are  identical. 

The  precise  definitions  of  these  morphisms  in  the  general 
context  is  given  in  chapter  5.  The  definitions  for  the  finite  and 
countable  case  are  in  chapters  1  and  2. 

The  vast  majority  of  the  present  work  is  concerned  with  the 
simplest  cases  of  the  morphisms  (between  a  MRP  and  a  renewal  process, 
and  between  two  finite  or  countable  MRP's).  The  reason  for  this  is 
twofold.  First  of  all,  as  far  as  applications  are  concerned,  these  are 
the  most  important  cases.  Also,  since  so  little  work  has  been  done  in 
this  area  it  seems  rediculous  to  jump  into  the  general  setting  without 
a  thorough  discussion  of  the  important  special  cases. 
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2.  Lumpabillty 

Probably  Che  simplest  case  of  equivalence  between  MRP's  Is  lump- 

ability  in  Markov  chains  [8].  Let  {X^}  be  a  Markov  chain  on  a  finite  or 

countable  state  space  S.  Let  A1tA0,--*,A  be  a  partition  of  S,  and  let 

1  i  n 

F:  S  -*■  {A^»A2»  •  •  • , A^}  be  the  map  that  "lumps"  the  state  space  S  onto  the 
partition  {A^.Aj, • • • .A^}.  The  process  {F(Xn>}  may  or  may  not  be  a  Markov 
chain.  In  general,  the  probability  of  going  from  A^  to  A^  in  {FCX^)} 
will  depend  on  precisely  which  element  of  A^  the  { X^ }  process  is  in.  If 
for  each  i  and  j,  though,  the  probability  of  going  from  A^  to  A^  is 
independent  of  the  state  in  A^  that  the  {X^}  process  is  fn,  then  the 
process  {F(Xn>}  is  a  Markov  chain.  When  this  happens  we  say  {X^}  is 
strongly  lumpable  to  {F(Xn>}.  This  is  a  special  case  of  the  equivalence 
to  be  defined. 

For  example,  say  S  =  {1,2,3}  and  let  {X^}  have  transition 
probability  matrix 


Let  F(l)  »  A^,  F(2)  -  F(3)  *  A2>  The  process  {F(Xn>}  is  a  Markov  chain 
on  (A^.Aj)  with  transition  probability  matrix 

/  I  A 

; 

\  2  2 
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If  {X^}  is  strongly  lumpable  to  { F ( > }  then  no  matter  which 

state  in  S  the  process  starts  in,  {F(X  )}  will  be  a  Markov  chain.  In 

n 

fact,  even  if  the  precise  state  that  the  process  begins  in  is  not  known, 

the  ensueing  (FCX^)}  process  is  a  Markov  chain. 

Sometimes,  even  though  {X^}  is  not  strongly  lumpable  to  {F(X  )}, 

the  process  { F (X  ) }  is  a  Markov  chain  when  {X  }  is  in  steady  state.  When 
n  n  J 

this  happens  we  say  {X^}  is  weakly  lumpable  to  { F ( X^ ) } . 

If  S  has  m  elements  (m  <  »)  and  F(S)  has  n  elements  A^,A0 , • • • , A^ 
(n  <  m) ,  then  the  following  m  x  n  matrix,  U,  can  be  constructed.  Let 

0,  if  i  A  , 

1,  if  i  e  Aj. 

There  is  a  vector  n  that  satisfies  nP  =  n  (the  steady  state 
vector)  where  P  is  the  transition  probability  matrix  for  i X  ) .  Let  !! 
be  an  n  x  m  matrix  with 

I  0,  if  j  t  At, 

,  if  j  e  Ai. 

The  ith  row  of  II  is  the  conditional  probability  of  being  in  state  j 

given  that  the  process  is  in  steady  state  and  that  the  process  is  in  A.. 

Kemenv  and  Snell  [8.],  show  that  {X^l  is  strongly  lumpable  to 

{F(X  )>  if  and  only  if  PU  =  U(IIPU),  and  that  if  {F(X  )}  is  a  Markov 
n  ^  n 

chain  then  its  transition  probability  matrix  is  IIPU.  They  also  show 
that  IIP  »  (IIPU)II  or  PU  »  U(KPU)  is  a  sufficient  condition  for  {X^}  to  be 
weakly  lumpable  to  {F(Xn)l.  For  example  let  S  =  {1,2,3}  and  set 
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F(l)  =  Ar  F(2) 


F(3)  *  Aj.  Suppose  the  transition  probability  matrix 


for  {X  }  is 
n 


1 

2 


1 

4 


In  this  case  ir  = 


,1  1  1, 

j*  3*  3'  s<^ 


0  0 


{X^}  is  not  strongly  lumpable  to  {F(Xn>}  since 


P(F(Xn)  =  A1|F(Xn_1)  =  A2) 


depends  on  whether  X^_^  is  equal  to  2  or  3.  This  can  be  seen  formally 
by  noting  that  PU  t  U(ITPU).  In  steady  state,  though,  {F(Xn)}  is  a  Markov 
chain  since  HP  =  (TIPU)  n.  The  resulting  Markov  chain  {FCX^)}  has  a 
transition  probability  matrix 


npu 


The  necessary  conditions  for  weak  lumpability  are  much  less 
appealing  than  the  necessary  and  sufficient  condition  for  strong  lump- 
ability  or  the  sufficient  conditions  for  weak  lumpability.  If  y  is  a 
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probability  vector  on  S  then  define  [y]*  to  be  the  vector  of  conditional 

probabilities  of  being  in  state  j  (j  = 1,2, • • • ,m) ,  given  that  the  process 

is  in  A^.  For  example,  the  iC^  row  of  the  matrix  II  is  [ir]*".  Let  r  be 

the  set  of  all  finite  sequences  of  states  in  F(S)  that  end  with  Aj.  If 

A,  ,  A  ,  •  •  •  ,A  ,  A.  and  A  ,  A  , • * • ,A  ,  A  are  two  elements  of  T.  then 
X1  12  1k  3  31  32  3q  3  3 

for  {X^}  to  be  weakly  lumpable  to  {F(Xn>}  it  must  be  true  that  for  each 


a  e  (1,  2, • • • ,n) . 


EE 

i  £  S  S  £  A 


pie  Yi 


EE 

i  e  S  6  e  A 


PiB  Yj 


where 

i  i,  i,  i,  1  , 

y  =  [  •••  1 1 [ l [it]  1  p]  p]  •••  P]  *  p]j 

and 

y2  =  [  •••  [[[[fir]jl  P]32  P]33  P]34  •••  P[3q  P]j  . 

Serfozo  [16]  showed  that  strong  and  weak  lumpability  can  be 

defined  for  MRP's  in  an  analogous  manner.  In  fact,  the  conditions  for 

strong  and  weak  lumpability  in  MRP's  are  virtually  indentical  to  the 

conditions  for  Markov  chains.  If  {X  ,T  }  is  a  MRP  on  a  finite  state 

n  n 

space  S  =  (1,2,3, ••• ,m),  with  kernel  Q(t) 

(i.*.  Qlj(t)  -  P(Xn+1  -  J,  Vl  <  t]Xn  -  i)> 

and  F:  S  -*■  (A^jA^,  •  •  • , A^}  is  a  partition  of  the  state  space  then 

(X  ,T  }  is  said  to  be  strongly  lumpable  to  (F(X  ),  T  }  if  (F(X  ),  T  } 
n  n  n  n  nn 

is  a  MRP. 

Again,  let  w  be  the  steady  state  vector  for  the  embedded  Markov 

chain  (i.e.  ttQ (°°)  =  x),  and  let  II,  U  be  defined  as  before.  Serfozo 

shows  that  (X  ,T  }  is  strongly  lumpable  to  (F(X  ),  T  }  if  and  only  if 
n  n  n  n 
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Q(t)U  -  U(nQ(t)U)  for  all  t  c  (0,<“]«  Likewise  if  for  all  t,  Q < t ) 1 1 
-  U(nQ(t)ll)  or  nq(t)  -  <nq(t>U>ll  then  {F(Xn>,  T  }  is  a  MRP  in  steady 
state  (i.e.  weakly  lumpable).  Unfortunately,  the  necessary  conditions 


for  weak  lumpabillty  are  again  very  complicated.  Let  be  the  set  of 

all  finite  sequences  of  states  In  F(S)  that  end  with  A  .  If  A  ,  A  , 

J  ij  i2 

•••.A  ,  A  and  A  ,  A  .•••,A  ,  A  are  two  elements  of  P.  and 

lk  ’  Jl  '2  • q  '  ' 

Itj,  t.,,  •  •  •  tfc  1,  {Sj,  s2»‘'’,8q^  aro  two  sequences  of  positive  real 

numbers  then  for  IX  ,T  )  to  bo  woaklv  lumpable  to  { F ( X  ),T  )  it  must  be 
n  n  -  n  n 

true  that  for  each  «  i  (1,2, •••,n)  and  t  r  (0,*"|, 


i  r  S  P  e  A 


'ip 


<l>  tj1  -  yi 

x  j  ..  r  fl  -  A  1  p  1 


1  r  S  6  r  A 


where 

l  l.  l,  l.  ‘z  k  , 

/  -  l  •••  11111*1  1  Qtt^l  1  Q(t2)]  J  Q( t.t))  U  •••  Q(tk  l)l  Q(tk)  1 1 


and 


"  l 


hum 


qfsji i 


tj  ( «  2 )  I 


q(s  ^  i 


In  this  paper  a  type  of  equivalence  will  be  defined  that  Includes 
all  ol  the  cases  discussed  so  far  and  has  the  added  property  that  a 
necessary  and  sufficient  condition  for  two  MRP’s  to  be  equivalent  can  be 
written  in  a  simple  form. 


1.  Definitions  and  Notation 

In  the  first  four  chapters,  all  MRP’s  will  have  finite  or  count¬ 
able  state  spaces.  l.et  S  be  a  finite  or  countable  subset  of  1R  ,  let 

*v 

denote  the  Borel  subsets  of  [0,n'|  and  let  il  »  (S  x  1 0 , «*  1 )  ,  the  countable 
cartesian  product  of  S  x  |0,w]  with  Itself.  Thus,  w  r  W  can  be 


represented  bv 


u*  -  llx^lui),  t  t.,0):  x  f*)  '  lMl‘»  •  1 0 ,  •>■  j ,  ii  -  0, 1  I . 

Let  X  (u»)  ■  x  luO  :iml  T  (u>)  -  t  (nO.  Lot  ii*  hi'  the  smallest 
n  ti  ti  » 

o-algebra  on  il  that  makes  tX  .  T  :  n  -  0,1, .’,•••!  random  variables,  and 
let  be  the  smallest  0-algebra  that  makes  IX^,  T^:  n  -  0, 1 . , •  •  • , k ) 

random  variables.  Let  1'  be  a  probability  measure  on  is.',  L 


Petlnition  1.  1.1.  The  process  tX^.  T  I,  n  -  0,1, is  a 
Markov -renewal  process  (MKP)  It  there  is  a  Q^it):  S  x  S  x  |0,."|  *  10,1 
such  that 

Vn.  TiX  -  1.  T  '  t|  tp  ,)  -  Qv  .it)  a . s . 

u- 1 

Oet  lull  Ion  I.T.*.  The  kernel  ot  a  MKl’  ts  the  matrix  ot  sub 

distribution  functions,  Q(t)  «  (Q^it):  1,1  >  S,  t  i  1 0 ,  •'*  1  ) . 

Lemma  l.  1.1.  It  IX  ,  T  1  is  a  MKl’  with  kernel  t  J  i  t )  then 
n  n 


T(\,  i  A,.,  •  •  •  ,X  i  A  ,  T,  v  t  ,  .  .  •  ,T  «•  t  > 
V  0  m  m  l  l  m  in 


I  A  |  i  A 
'll  0  n  n 


Proof.  01  eavlv  l'tX(1  r  A^) 


hypothesis  is  true  tor  k  «•  in.  Then, 


I’iX^  -  1  ).  Assume  the 


r<\,  '  V”'-*. '  *„•  Ti  '  'v"  -\  ' 


*  2  ]£  P'*>'  '  V" ’ 


I  t  A  I  A  . 
Jm  m  tn-1  1 
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-E  E 

j  cA  j  ,eA  , 
m  m  m-1  m-1 


P(Xm  “  jm'  Tm  -  SJVl  "  Jm-1’  Xm-2  e  Am-2’ 

•  ■  * » xo e  A0 *  ViiVi.--Tiih) 


P(Xm-l  “  Jm-1*  Xtn-2  £Am-2’'",X0  c  A0’ 


T  <t  •  •  •  T  <t^ 

m-1  -  Vl’  -  V 


Ey  P(x  -  j  ,  t  <  t  lx  -  i  .) 

m  Jm  m  —  m1  m-1  m-1 


j  eA  j  eA 
m  m  m-1  m-1 


z  z 

j0eA0  Jm-2eAm-2 


■  Z  Z  p<xo 

JoA0  \A„ 


j0)Qj0j1(tl)' 


•Q,  1  (tJ- 

jm-ljm  m 


□ 


Let  tJt/ be  the  semi  algebra  of  sets  of  the  form  (X_ e Art,  •  •  •  ,x  eA  , 

°  0  0  n  n 

T.  <  t,,*'*,T  <  t  ).  For  A  tiV,  let  P(A)  *  P(A).  Since  P  is  the 

1—  I  n  —  n  ’ 

extension  of  P  from  A  to  ^it  is  clear  from  the  lemma  that  the  kernel, 

Q(t),  and  the  initial  distribution,  y  -  { y j  :  ]  e  S)  (where  y^  “P(Xq-  j)) 

completely  specify  the  distribution  of  {X  ,  T  },  n  *  0,1,2,--’.  For  this 

n  n 

reason,  throughout  this  paper  a  MRP  will  be  specified  by  its  kernel  and 
initial  distribution. 

Definition  1.3.3.  A  MRP,  (X  ,  T  },  n  •  0,1,2,***,  with  kernel 
-  n  n 

Q(t)  and  initial  distribution  y  will  be  denoted  (Q,y).  A  MRP  with 
kernel  Q(t)  and  unspecified  initial  distribution  will  be  denoted  Q. 

Occassionally ,  we  will  need  to  consider  the  individual  sample 


paths  of  the  MRP  (Q,y).  This  poses  no  problem  since  we  can  define 
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df)  as  before  and  construct  P  by  extending  P. 

If  S  has  only  one  element  then  il  -  (0  %  t»  )  ^  &-i£m  and 
“ tl6  .  Thus,  if  (X^,  T  ) ,  n  ■  0,1,2,* •*,  is  a  MRP  with  only  one 
state 

»’(Tn  -  t|  <fn_v)  -  P(Tn  -  t)  -  r(t)  a .  s . 

Thus,  {TpT,,***}  Is  a  sequence  of  i.i.d.  nonnegative  random  variables 
with  common  distribution  r(t). 

Definition  l.  1,4.  A  MRP  with  one  state  will  be  called  a  renewal 
process .  The  kernel  ot  a  renewal  process  Is  a  scalar  tunc (ion,  r(tl. 
Since  there  can  he  no  confusion  about  the  initial  distribution  ot  a  MRP 
with  only  one  state,  a  renewal  process,  lTj,T.,,***}  with  P(T  *•  1 1  «  r(tl 
will  be  denoted  r. 

Let  (Q.y)  be  a  MRP.  By  renumbering  the  states,  the  kernel  i,'(t) 
can  always  be  put  into  the  following  canonical  form.  (see  Clnlar  l  1 1 > 
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where  the  columns  of  Q j  C  t )  correspond  to  the  rows  ot  (interpret 

J+l  -  1  if  j  -  n).  If  n-1  (i.e.  Q(t)  -  Q^(t)),  the  MRP  is  called 

aperiodic,  otherwise  it  is  periodic  with  period  n. 

Notation.  U  will  always  denote  a  column  vector  of  l's. 

ftgfinltion  1.3.5.  If  Q(t)  is  a  kernel  of  a  MRP  then 

Q(“>)  -  lim  Q(t). 
t-*“ 

Since  Q(t)  is  right  continuous  it  is  possible  that  for  some  i, 

<  l.  This  means  that  if  ^  *  i  there  is  a  positive  probabil- 
J 

ity  that  T  -  This  can  be  Interpreted  as  a  process  that  terminates  at 

time  T  ■  T.  +  T  +  •  •  •  +  T  where  n*(w)  "  Inf  {T  (u>)  ■  «“}.  It  is  well 
i  j  n w  n*rl 

n 

known  (^inlar  (3))  that  if  Q  is  irreducible,  P(n*“<»’)  -0  if  <  l' 

and  P(n*“«’)«l  it"  Q(<»)U  -  U. 

Definition  1.3.6.  An  irreducible  MRP  is  said  to  be  transient  if 
P(n**»)«0  and  presistent  if  P(n*  “«»)••  l .  Note  that  a  transient  MRP 
and  a  MRP  with  transient  states  are  different  concepts.  For  example, 
if  r(«>)  -  1  then 


r(t  1 

r(t ) 

_ 

is  not  a  transient  MRP  even  though  every  state  is  transient. 

In  this  paper  irreducible  MRP's  will  be  persistent  unless  stated 
otherwise. 

It  is  well  known  (Cinlar  |3])  that  It  Q(»)  is  an  irreducible 
matrix  there  is  a  unique  positive  eigenvector  associated  with  the  larg- 


i 


esc  eigenvalue  of  Q(“). 

Definition  1.3.7.  Let  Q  be  an  irreducible  MRP.  Let  n  satisfy 
tu)(")  »  Xn  and  nl!  »  l  where  X  is  the  largest  positive  eigenvalue  of  i)(®)  ■ 
n  will  be  called  the  steady  state  vector  for  Q. 

4.  Elementary  Facts  about  MRP*  s 

Let  (Q,v)  be  a  MRP.  We  will  need  to  work  with  equations  involv¬ 
ing  terms  like 

(1)  PCq  <  t2  <  t2.-.-,Tn  t  tn.  xn  -  j) 

(2)  P(Tj  .<  tr  T2  <  t2.'".Tn  i  tn|X0  -  i) 

(3)  P(T1  <  tj.  T2  <  <  t„). 

Fortunately,  from  lemma  1.3.1,  these  quantities  can  be  written 
down  in  an  attractive  form  using  the  matrices  (i)(t)},  t  e  [0,«|,  and  the 
vector  y. 

< l * )  P(T1<t1,  r2<  t2.-".Tn  < ;tn,  xn  -  j)  -  (YQ(tl)Q(t2)  — Q(tn)) j 

(2')  P(T1<t1,  T2  <  t2 .  *  -  • ,  Tn  <_  tn  I  Xq  -  i )  -  W(tl)Q(t2)-*-Q(tn)U)t 

(3')  PfTj^tj,  T2  <  t2,..*,Tn<  tn)  -  YQ(t1)Q(t:)---Q(tn)U, 

Let  Q(t)  be  the  kernel  of  a  finite  state,  irreducible,  aperiodic 
(possibly  transient)  MRP.  By  the  Perron-Frobenius  Theorem  (3)  there  is 
a  unique  largest  eigenvalue  of  Q(“)  which  is  positive,  and  that  eigen¬ 
value  has  an  associated  left  and  right  eigenvector  which  have  positive 
elements.  Let  X  be  the  eigenvalue,  *  the  left  eigenvector  and  S  the 
right  eigenvector.  Assume  that  it  is  normalized  so  that  t>u  -  l  and  S  is 
normalized  so  that  ttS  -  1.  Since  X  has  multiplicity  one,  we  can  write 
Q(«*)  ■  AJA  *  where 
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A 

IT 

J2 

J3 

A  * 

B 

B 

> 

1 

)— * 

1 

c 

,Jk 

* 

» 

Since  A  is  the  largest  eigenvalue. 


lira  A~n  Jn 


l 

0 

0 

0 

Thus, 


lira  A-nQn(»)  -  A(lim  A  njn)A_1  -  Bit. 


In  the  case  where  (Q.y)  is  persistent,  A-l,  B-U,  and  we  get 

the  well  known  result  that  lira  Qn(»)  *  Un. 

n-*» 

If  Q(«°)  has  period  m  (i.e.  has  form  1.3.2)  then 


Qm(®)  .  §2 


where  Qj  *  '  *^j-l  *  ^ach  Qj  irreducible  and  has  the  same 

maximal  eigenvector  A.  Thus  if  tt^  and  Bj  are  the  normalized  left  and 
right  eigenvectors  associated  with  A  then 


L 


«  »  «  “  run  _  run  <  » 

lim  A  Q  (®) 


Furthermore 


,  if  it  -  (it  ,  )  and  6  =  (3,,  )\  then 

X  Z  m  ±  z  m 


, ,  1  -(nm+k)  nm+k. .  „ 

lim  ~  >  A  Q  (»)  =  Bit. 

m  /  j 

n4a) 


Let  Q  have  the  general  form  (1.3.1)  and  assume  now  that 
q(oo)U*u.  Say  Aj  has  period  and  let  m  be  the  least  common  multiple 
of  the  ®j’s*  Since  each  is  a  transient  sub  MRP,  we  get 


1  \  1  .nnrt-k ,  . 

;2^Q  (*) ' 
k*l 

Utt 

n 

hl^l 

P121t2 

. . . 

P%  * 
In  n 

P21*l 

P22Tt2 

p-> 11 
2n  n 

• 

• 

’■ 

• 

• 

Pdl’l 

Pd2Tr2 

. . . 

P  .  * 
dn  n 

where  p^j  is  the  probability  of  ending  up  in  Aj  given  the  process  starts 
in  the  ith  transient  state,  (see  (3]). 


mm 


CHAPTER  II 


EQUIVALENCE 

1 .  Equivalence 

Associated  with  each  MRP  (Q,y)  is  a  sequence  of  nonnegative  (pos¬ 
sibly  extended  valued)  random  variables  (TpT., ,•••).  The  distribution 
of  {T. , is  given  by 

P(T1  *-  lr  T2  '  1 2 • " "  ' 1  Tn  <  ”YQ(t1)Q(t2)---Q(tn)U. 

Lemma  2.1.1.  (T..T,,***)  is  a  sequence  of  i.i.d.  nonnegative 

random  variables  (i.e.  a  renewal  process)  if  and  only  if 

Vn,  tj,  t2.  •  •  • .  tn,  YQO^KHtJ  •  •  -QU^U  -  (yQU^U)  (yQU^U)  •  •  •  OQU^U) . 

Proof .  (—*)  If  is  a  renewal  process  then 

P(T1  -  V  T2  <  c2*  *  ‘  *  ,Tn  v  tn)“P(Tl  <  tl)P(Tl  v  ‘^---PlTj  v  t^). 

But  this  says  yQ(t1)Q(t ,)  •  •  *Q(tn)U  »  (yQd^HO  UQ(t  ,)U)  •  •  •  C>Q(t  )U) . 

(«—)  if  YQ(t1)Q(t2)---Q(tn)U  -  (>Q(t1)U)(>Q(t2)U)-- •(KUtn)U) 
then  PC^  <  t1,  T2  <  t2,***,Tn  <  t  )  -  P(Tj  <  tjlPO^  v  t^-'-Pdj  <  t^)  .>  ’ 
This  leads  to  the  definition  of  equivalence  between  a  MR!'  and 
a  renewal  process. 

Definition  2.1.1.  Let  (Q,y)  be  a  MRP  and  let  r  be  a  renewal 
process.  (Q.y)  Is  equivalent  to  r  (written  (Q,y)  ~  r)  if 

Vn.  YQU1>Q(t2>-,-Q<tnHl  -  r(t  jM^)  •  •  -rltj  . 
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If  for  any  Initial  distribution  8,  (q,8)  -  r  then  we  write  Q  ~  r.  If 
there  exists  an  initial  distribution  8,  such  that  (Q,8)  ~  r  then  Q  ~  r. 

Clearly,  if  (q,y)  ~  r  it  must  be  true  that  r(t)  =  yQ(t)U.  The 
main  questions  that  will  be  answered  in  Chapters  II  and  III  are 

(1)  When  is  (Q,y)  ~  r? 

(2)  For  a  given  MRP  Q,  when  is  it  impossible  to  find  an  initial 
distribution  y,  and  a  renewal  process  r  that  yields  (Q,y)  ~  r? 

(3)  If  (Q.y)  —  r  and  (Q,8)~f  when  must  r  =  f? 

(4)  If  (Q.y)  ~  r  and  (Q.8)  ~  r  when  must  y  =  8? 

2 •  Conditions  for  Equivalence 

In  this  section  we  give  sufficient,  necessary,  and  necessary  and 
sufficient  conditions  for  (Q,y)  ~  r. 

Theorem  2.2.1.  Let  (Q.y)  be  a  MRP  and  let  r  be  a  renewal  pro¬ 
cess.  If  Vt,  yQ(t)  -  r(t)y  then  (Q,y)  ~  r. 

Proof.  If  V t,  yQ(t)  -  r(t)y  then  Vn,  tj.t,, • • • ,tn« 

yq(t1)q(t2)---Q(tn)U  -  r(t1)yQ(t2)Q(t3)---Q(tn)ll 

«  r(t1)r(t2)yQ(t3)---q(tn)U 

*  r(t1)r(t2)**'r(tn)yU 
■  r(t1)r(t2) • * ’ r (in) •  <-) 

Theorem  2.2.2.  IfVt,  Q(t)U-Ur(t)  then  Q  a  r. 

Proof.  IfVt,  Q(t)U-Ur(t)  thenVn,  tj,  t2,“'tn 

8Q(t1)Q(t2)*--Q(tn)U-  8Ur(t1)r(t2)---r(tn)  -  r(t1>r(t2> • • • r(tn> 


for  any  initial  distribution  8.  a 
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Theorem  2.2.1  and  2.2.2  are  special  cases  of  the  sufficient  con- 

m 

ditions  for  weak  lumpability  that  Serfozo  gives  in  [16].  Theorem  2.2.1 
says  that  if  y  is  a  left  eigenvector  of  Q(t)  for  every  t  then  (Q,>)  -  \ 
where  r(t)  is  the  eigenvalue  of  Q(t)  corresponding  to  the  eigenvector  > . 
Theorem  2.2.2  is  a  special  case  of  the  necessary  and  sufficient  condition 
for  strong  lumpability  in  Serfozo  [16].  Theorem  2.2.2  says  that  if  the 
row  sums  of  the  matrix  Q(t)  are  the  same  for  every  t  then  Q  &  r  where 
r(t)  is  the  common  value  of  the  row  sums.  If  the  row  sums  are  all  the 
same,  the  times  between  transitions  do  not  depend  on  the  state  of  the 
process.  Thus 


P(T  <  t  |X  ,,X  „ ,  • • •  X- ,  T  ,  ,  •  •  •  ,T,  )  “  P(T  <  t  |X  ,  )  =  P(T  v  t  ) 

n  -  n1  n-1  n-2  O’  n-1  1  n  -  n1  n-1  n  n 

so  {T^.T^, • • • .T^}  a  renewal  process.  The  intuitive  justification  of 

theorem  2.2.1  is  less  obvious,  but  most  interesting  cases  of  equivalence 

seem  to  be  of  that  type.  It  will  be  shown  in  section  four  that  Burke's 

theorem  is  a  corollary  of  theorem  2.2.1. 

In  everything  that  follows,  the  topology  on  lRn,  n  <  will  be 

the  L^  topology  (i.e.  if  x^  c  lRn,  K  =  1,2,  •  ••  then  x^  -*■  x  if  x^U  ►  xU) . 

Let  (Q,y)  be  an  n  state  MRP  (n  <  »)  and  let  r  be  the  renewal 

process  with  distribution  yl}(t)U.  We  define  the  following  subsets  of 

]Rn. 


Definition  2.2.1. 

(A)  Let  *  Iv  f  lRn:  vQ(t)U  -  (vU)r(t),  Vtl. 

(B)  Let  be  the  largest  subset  of  that  is  invariant  under  multi¬ 

plication  by  Q(t)  (i.e.  J^Q(t)  C  Jf  ,  Vt). 

y  > 

(C)  Let  dP  =  {v  t  ®n:  v  >  0,  vU  *  11.  dP  is  the  set  of  probability 
vectors. 

(D/  Let  n  dP , 
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Lemma  2.2.3.  V  is  well  defined. 
-  Y 


Proof.  Let  <.A  be  the  class  of  all  subsets  of  A  that  are 


invarient  under  multiplication  by  each  Q(t).  Let  {A^},  n  t  1  be  an 

Increasing  chain  in  d  .  Since  UA  is  invariant  and  is  an  upper 

a 

bound  for  the  chain,  there  is  a  maximal  element  of  (Zorn's  Lemma) .  □ 


Lemma  2.2.4.  Ay  and  are  subspaces  of  lRn. 


Proof.  Clearly  {y.0}  C^Y  ,  SO  Ay  has  at  least  two  elements. 


Let  01,  02  e  Ay'  Then  61Q(t)U  =  rUH^D)  and  02Q(t)U  =  r(t)(02U). 
Thus  (a01 +b02)Q(t)U  =  r(t) (a01 + bg,)U,  so  agj+bg.,  t  d  .  Thus^^  is 


a  subspace. 


Let  W  be  any  invariant  subset  of  and  let  be  the  subspace 

k  T 

generated  by  m.  Say  w  c  w*.  w  =  aiwi  w^ere  eW  .  But 

k  k  11 


wQ(t)  *  V  a.w.Q(t)  =  E  f,w,'  where  w.'  since  ^  is  invariant. 

ftl  1  1  ft  1  1  1 

Thus  wQ(t)  etyf*  so  is  invariant.  Since  is  defined  to  be  the 


largest  invariant  set  it  must  be  a  subspace.  O 


Lemma  2.2.3.  is  closed,  convex  and  invariant  under  multi¬ 


plication  by  Q(t)/r(t),  Vt.  If  n  <  OO,  CHy  is  compact. 


Proof .  Since  is  a  subspace,  Cfiy  is  closed  and  convex.  Let 


B  c  .  Since  C  *{/ ,  0Q(t)  =  r(t)0'  where  P'  e  %/  ,  and  since 
y  Y  Y  Y 


YyCjAy  •  BQ(t)U  =  (0U)r(t)  =  r  (t) .  Thus  P'U=1  so  P'  r  CK  y  •  Since 


is  closed  and  convex  it  is  compact  whenever  n  <  <*>.  D 


Corollary  2. 2. 5.1.  If  Q(°°)  is  irreducible  and  persistent,  the 


steady  state  vector  tt  Z<Xy' 


Proof .  Say  Q(tD)  has  period  m.  Then 

,  n+ra-1 


lim 


n-xo 


(«>)  =  Utt. 


n-Hn-1 

Vn  ct#  ,  Pdim  l  T\  QJ(“))  -  n. 

Y  rv*-  m  fsH 

Since  <%y  la  closed  and  convex,  u  £K  y  .  n 

Consider  the  column  vector  Q(t)U.  If  (^  ,  f*2  E  ,yc  then 

2^Q(t)U  *  P  2Q ( t )  U ,  Vt  which  says  (P^  ~  P2)  orthogonal  to  <}(t)U,  Vt. 

Lemma  2.2.6.  If  (Q,y)  Is  an  n  state  MRP  (n  <  “>)  and  there  exist 

times  tj»t2’"**,tn  suc*'  that  {Q(t^)ll,  t}  ( 1 2 )  ^  * '  ‘  ‘  ,  Q  ( t  )  U }  Is  a  linearly 

independent  set  then  Is  the  subspace  of  IR  n  generated  by  ly). 

Proof.  If  p  r  then  (  ~  -  y)  1  Vt.  Hut  only  t hi*  zero 

Y  pl> 

vector  can  he  orthogonal  to  n  independent  vectors  in  IRn.  Thus,  the  only 
elements  of -Cjf  are  the  points  on  the  line  through  y  and  the  origin,  n 
We  now  show  the  importance  of  the  set  ar . 

Theorem  2.2.7.  (IJ,  y)  ~  r  1!  ami  only  if  y  r CH y- 

Proof.  (  ~=**)  If  (Q,y )  ^  r  then  Vn,  t,,t_,***,t  , 

yQ(t ^)Q(t2> ’ • *Q(tn)U  »  r(tj)r(t2)***r(t  ).  Sav  there  exists  tj,t2,-*’,tk 
such  that  YQ(t  j)Q(t2)  •  •  *(}(tk)  .  Then  there  is  a  t  such  that 

YQ(t1)Q(t2)**-Q(tk)Q(t)U  i  (yg(t1)Q(t2)---Q(tk)U)r(t).  But  this  says 
yQ(t^)Q(t2) • • ‘Q(tk)Q(t)U  i  r(tj)r(t2)* • *r(tk)r(t )  which  contradicts 
(Q,y)  ~  r.  Thus  Vk,  t^t^*-*,^  we  have  yQ(t  j  )  tj(  1 2 )  •  •  *  tj  <  t  k )  t 
bet  *  (y(U  (w:  w  »  yQ(t^)Q(t2) • • *Q(t^)  for  some  n,  tj, t2, • • • , t^ ) . 

m  C  and  m  is  invariant  under  multiplication  by  Q(t)  so  y/. 

Since  y  *<9(rn&  we  have  y  c  ,ycr 

(4—)  if  Y  rt7(  then  Vn,  1  \  • 1  2  •  ’  ’  ’  • 1  n*  yg(t ,  )t)(t  2>  “  'l^ln^ 

-  r(ti)r(t2)  •  •  •  r(tn)y’  where  y'  c  Thus  y<7(  t  j  K>{  1 2 )  •  •  •<)(  t  n>l) 


•*  *  id  l  V 


. . .  rl°  *  *  -  r  . .  lr<Jr  ,  t. 


(  — »>  »v  (.'o  roll  .1  tv  J.J.s.l.  1,  ( 

. .  W”'  '  '  Iiv  (!».,. 

^ ^  **  **  $  t ho ii  n  i  ,ys  »>v 

«  '  (‘Mollniy  «  .so 

-  r.  i) 


-i'll 


<  .  .  „  -  (>  '  t) 

’  J  •  1  (,  •  j  •  2  ’  • 


r(t ) 


Claim:  . 

Proof . 


»Q(t) 


_  J 


r(i  >>  it  t  v  t** 


r(t)i  It 


t  '  i 


A* 


0  ll  t  v  t* 

!  H  t*  .  ,  v  ,** 

l  HI-  t*\ 


vQ(t) 


j  'U)y 
^  r(t ) n 


It  t  v  t** 

1 1  t  ■*  i**. 
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Thun,  V«,  f2’ '  * "  tn*  1,Q(t1KMt2)  •  •  •  g ( t^>1)  -  r ( 1 1 ) i (t,)  •  •  -r(tn). 

Clearly  (Q,y)  ~  r  also.  Note  that  Q(t)ll  4  r(t)U  if  t*  <  t  <■  t**, 
nQ(t)  i  r(t)n  If  t  <  t**  ami  yQ(t)  4  r(t)y  if  t  *  t**  so  the  conditions 
of  theorem  2.2.1  and  2.2.2  are  not  satisfied.  In  this  example 

-X,  -  . x ,  -  *• 

Theorem  2.2.1  and  2.2.2  give  sufficient  conditions  for  ((),>)  r 
wltich  are  relatively  easy  to  use  in  practice.  Theorem  2.2.7  gives  a 
necessary  and  sufficient  condition,  hut  there  Is  not  yet  any  simple  wav 
to  determine  what  '  is  in  general.  The  following  theorems  give  neces¬ 
sary  conditions  for  (i),y)  ~  r  which  are  also  useful  in  practice.  In 

everything  that  follows,  *  0  if  r(t)  -  0. 

r  ( t ) 

Theorem  2.2.8.  If  (Q,y)  is  a  finite  state  MRP  and  (Q.>)  ~  r  then 
r(t)  is  an  eigenvalue  of  t)(t)  for  each  t.  If  in  addition  Q(t)  is  Irrcduc- 
Ible  for  each  t,  r(t)  is  the  largest  eigenvalue  of  Q(t). 

Proof.  If  (Q,y)  is  a  finite  state  MRP,  ^  is  compact  and  con¬ 
vex  and  satisfies  SH  0(t)C  r(t)  ,  Vt.  Thus,  hv  the  Brouwer  fixed 

Y  f 

point  theorem,  for  each  t  there  is  a  ^  {  such  that  >  t}( t )  -  r(t)>t<  so 
r(t)  is  an  eigenvalue. 

Say  Q(t)  is  irreducible.  Assume  \  Is  the  largest  eigenvalue  ol 
Q(t)  (which  must  he  positive)  and  \  "  r(t).  Thus  the  largest  eigenvalue 
of  Q(t)/r(t)  is  strictly  greater  than  one.  This  implies  that 
yQ(t)/r(t))nl>  diverges.  But  if  (I),  y)  ~  r  then  >  (Q(t)/r(t))Ul'  "  1  for  all 
n,  so  r(t)  must  be  the  largest  eigenvalue. it 

Cxamplo  2.2.2.  If  (Q.y)  is  an  infinite  state  MRP  and  (Q,y)  ~  r 
it  may  be  that  r(t)  Is  not  an  eigenvalue  of  Q(t).  For  instance,  sav 
0  <  t*  <  t**  and 


0 


If  t  <  t 


whether  r(t)  ia  an  eigenvalue  of  Q(t)  for  each  t,  theorem  2.2,8  says 
that  one  can  show  that  Q  f  r  by  merely  finding  a  value  of  t  where  r(t)  Is 
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not  an  eigenvalue.  The  following  theorem  Is  useful  In  the  same  way. 

Theorem  2.2.9.  If  (Q,y)  is  an  n  state  MRT  (n  <  ®)  and  there 
exists  times  such  that  (Q(t1)U,Q(t2)U,  •  •  •  ,Q(tn)U)  is  a 

linearly  Independent  set  then  (Q,y)  ^  r  if  and  only  if  V  t,  yQ(t)*r(t)y. 

Proof.  By  lemma  2.2.6,  is  a  one  dimensional  sufcspace  so 
either  or  W  »  {0}.  By  theorem  2.2.7,  (Q,y)  ~  r  implies 

Y  c,^Y,  so  must  consist  of  the  single  vector  {y}.  Since  X  is 

Invariant  under  multiplication  by  Q(t)/r(t),  we  have  yQ(t)  =  r(t)y,  Vt. 
The  converse  is  a  restatement  of  theorem  2.2.1.  n 

Corollary  2. 2. 9.1.  If  Q  is  an  n  state  irreducible  MRT  (n  <  «*) 
with  steady  state  vector  n  and  there  exists  times  t'K,t 

{Q(t^)U,Q(t2)U,  •  •  •  ,Q(tn)U}  is  a  linearly  independent  set  then  Q  r  if 
and  only  if  (Q,u)  ~  r. 

Proof .  tf  Q  is  irreducible  and  (Q,y)  ~  r  then  (Q,n)  ~  r  also. 

But  the  theorem  says  that  at  most  one  initial  distribution  can  yield  a 

renewal  process.  Thus  either  (Q,n)  ~  r  or  Q  i~  r.  o 

A  Rough  ^ljiotrJLthm  for  Determining  Whether  ~  r. 

In  general,  a  MRP  (Q,y)  is  not  equivalent  to  the  renewal  process 
yQU.  In  fact,  for  a  given  MRP,  Q,  there  is  usually  no  initial  distribu¬ 
tion  that  yields  a  renewal  process.  In  many  queueing  and  other  processes 
there  are  random  processes  that  can  be  easily  characterized  as  MRP's,  but 
the  interesting  and  important  question  is  whether  or  not  the  random 
process  is  a  renewal  process  (see  the  examples  in  the  next  section). 

Unless  there  is  a  very  good  reason  to  believe  that  the  process  is 
renewal  it  makes  sense  to  first  try  to  show  that  there  Is  no  initial 
distribution  that  yields  a  renewal  process.  If  Q  is  irreducible  (as  it 
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usually  will  be  in  practice)  the  simplest  and  most  successful  approach 
is  to  use  corollary  2. 2. 5.1  and  the  fact  that  if  y^,  Yj  ^  then 

]  Q(t)U,  V t  is  a  necessary  condition  for  (Q,y)  ~  r.  Corollary 
2.2. 5.1  says  that  if  Q  is  irreducible  with  steady  state  vector  ir  then 
n  e  +//  whenever,#  +  <J>.  Since,#  is  invariant  under  multiplication 

cr*'Y  y  y 

by  Q(t)/r(t)  we  must  have  (it  -  ^^-)Q(x)U  -  0,  Vt,  x.  If  Q  ~  r  there 
is  often  a  simple  argument  showing  one  will  not  get  zero  for  all  values 
of  t  and  x  (e.g.  example  2.4.3  in  the  next  section).  Otherwise  (as  in 
example  2.4.4)  the  multiplication  has  to  be  carried  out.  Of  course  it 
is  possible  that  (it  -  ^r^ip-)Q(x) ll“0,  Vt,  x  even  if  (Q,n)  1°  r  since 
vQ(t)Q(x)U  *  r (t ) r (x) ,  Vt,  x  is  not  a  sufficient  condition  for  (Q.n)  ~  r. 
An  example  where  this  occurs  would  be  very  difficult  to  construct, 
though,  and  it  is  very  doubtful  that  one  would  ever  come  across  one  in 
practice.  If  this  multiplication  appears  to  be  difficult  it  may  also  be 
possible  to  show  that  Q  *  r  by  showing  that  r(t)  is  not  an  eigenvalue  of 
Q(t)  for  some  t.  (see  example  4.2.1) 

If  the  tests  to  show  chat  Q  *  r  fail,  there  is  every  reason  to 
believe  that  Q  ~  r.  To  verify  that  Q  ~  r  it  makes  sense  to  show  that 
nQ(t)  -  r(t)n.  Vt  where  x  is  the  steady  state  vector  or  that  Q(t)U  ■ 

Ur (t) ,  Vt  (theorems  2.2.1,  2.2.2).  If  Q  has  n  states  (n < -)  and  {Q(tJ)U> 
J  -  l,2,"*,n  is  a  linearly  independent  set  for  some  *  ’ '  »tn’ 

theorem  2.2.9  says  that  i f  y  1*  it  and  xQ(t)  ”  r(t)n,  Vt  becomes  a 

necessary  (and  sufficient)  condition  for  Q  ~  r.  (see  example  2.4.1). 

4 .  Examples  and  Applications 

Example  2.4.1.  Disney,  Farrell  and  DeMorais  [4]  show  that  the 
output  of  an  M/D/1/1  queue  in  steady  state  is  a  renewal  process.  The 
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results  obtained  thus  far  allow  for  a  quick  verification  of  that  fact. 

The  output  from  an  M/D/l/1  queue  is  a  two  state  MRP  with  kernel 


Q(t) 


^00^  %1^ 


Q10(t)  Qll(t) 


where  Q^j(t)  probability  that  given  a  customer  departs  (at  time 

zero)  leaving  i  customers  in  the  queue,  the  next  departure  occurs  before 
time  t,  and  when  that  customer  leaves  there  are  j  customers  left  in  the 
queue. 

If  arrivals  are  Poisson  with  rate  X  and  the  service  times  are 

deterministic  with  duration  d  then 

-Ad.  -A(t-d),  -Ad, .  -X(t-d) ,, 

e  (1-e  )  (1-e  ) (1-e  ) 


Q(t) 


-Ad 


1-e 


-Ad 


1,(0, 


where  ld(t) 


0  if  t  <  d 


1  if  t  >  d. 


The  embedded  Markov  chain  has  transition  probability  matrix 


Q(») 


-Xd 

n  -Xd 

1-e 

-Xd 

,  -Xd 

1-e 

,  -Xd 

1  (e 

.  -Ad, 
1-e  ) 

plication  xQ(t)  we  get 

wQ( t)  -  (<l-e~U)e"Xo,  (l-e_/u-)(l-e_''u))ld(t)  -  (l-e'“)  ld(t)n. 
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By  theorem  2.2.1  we  know  (Q,ir)  ~  r  where  r(t)  ■  (1-e  )l.(t). 

d 

It  is  easily  verified  that  for  any  t^  j4  t£  the  vectors  Q(t^)U 
and  t; ( ^ ) 11  «re  linearly  Independent.  Thus,  by  theorem  2.2.9,  (Q,y)  ~  r 
if  and  only  if  yQ(t)  ■  r(t)y  for  each  t.  By  the  Perron-Frobenius 
theorem  we  must  have  y  =  it.  Thus  tt  is  the  only  initial  distribution 
that  yields  a  renewal  process. 

Example  2.4.2.  Burke's  theorem  [2]  implies  that  the  output  from 
a  steady  state  K/M/l  queue  is  a  Poisson  process.  The  output  process  is 
a  MRP  with  kernel  Q(t)  where  ^ ( t )  has  the  same  interpretation  as  in  the 

first  example  except  that  in  this  case  i  and  j  range  over  all  the  non¬ 
negative  integers.  If  the  arrival  rate  is  X  and  the  service  rate  is  p, 
the  kernel  has  the  form 


Vc) 

q1(c> 

q2U> 

f0(t) 

/"N 

rr 

f2(t) 

E0<t) 

fx(t) 

f0(t) 

where 


fj(t)  ■  ^J*  e  *S  ue  ySds  and 


q^(t)  -  ^  Xe  Sf j (t-s)ds. 


It  is  well  known  that  the  steady  state  vector  for  this  MRP  is 
2 

v  *  (l-p)(l,p,p  ,*••),  where  p  *  X/p.  Performing  the  multiplication 
»Q(t)  we  get 
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j 

(irQ(t))  -  (1-P)'1(q.(t)  +  £  f .  v(c>Pk+1> 

J  J  k-0  3 

-  (1-e  At)(l-p)  =  (1  -  e~^t)rTj . 


Thus,  the  steady  state  output  is  a  renewal  process  with  distribution 
1-e  (a  Poisson  process). 

Example  2.4.3.  Now  consider  the  M/M/l/N  queue  (N  <<*>).  The 
output  from  this  queue  is  an  N  +  1  state  MRP  with  kernel 


V'> 


q0(t) 

qx(t) 

q2(t) 

'  •  '  Vl(t) 

f0(t) 

f^t) 

f2(t) 

’  •  •  fN-l(t) 

fQ(t;) 

fx(t) 

fN-2^l) 

fo(t) 

‘  •  *  fN-3(t) 

J-N  ’ 

CD 


j=N-l 


j=N-2 


f„(t) 


6(t! 


where  C t )  and  fj(t)  are  as  in  the  Iasi  example.  The  steady  state 

N  1  -  p  , ,  N. 

vector  is  n  - - r  (1,P,***,P  )• 

,  N+l 

1  -  p 

If  the  steady  state  output  from  the  M/M/l/N  queue  is  a  renewal 

process  then  the  renewal  process  would  have  to  have  distribution 

N  -Xt  N+l  -pt 

rN(t)  -  n  QN(t)U  1  "  N+l  +  ^  n+T  ‘ 

1  -  p  1  -  p 
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N  v  QN(x) 

If  u  e  tX  (for  some  y)  then  so  is - 7— c-  for  any  x.  Thus, 

Y  rN(x) 

N  n\(x) 

Vt,  x.  (x  -  --  yv\~  )  1  q(t)U. 

V  ' 

“Ax 

Clearly  Q^(t)U  has  the  form  (a,b,b, • ■ • ,b)  and  r^(x)  <  1-e 


Also,  since  nNQN(x)  -  — (1-e  Ax) (l,p, • • • ,pN  ^Kfx))  where 

1-p 


K(x) 


rN(x) 


-Ax 


1-P 


N+l 


1-P 


a  straight  forward  calculation  yields 


(u 


N 


xV(x) 

- .  )  Q(t)U  i  0  for  any  t,  x  >  0.  By  corollary  2. 2. 5.1,  the 

rN(xl 


output  from  an  M/M/l/N  queue  is  not  a  renewal  process  for  any  initial 
distribution. 

Example  2.4.4.  Consider  the  following  Jackson  network. 


There  is  a  Poisson  arrival  stream  with  rate  A,  an  exponential 
server  with  rate  y,  and  a  Bernoulli  switch  that  feeds  a  departing  cus¬ 
tomer  back  to  the  end  of  the  line  with  probability  p.  The  input  process 
is  defined  to  be  the  superposition  of  the  arrival  process  and  the  feed¬ 
back  process,  (see  [5]  for  a  complete  discussion  of  this  problem).  The 
input  process  is  a  MRP  with  kernel  Q(t)  where 
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v° 


|0  if  j  >  1  +  1 


jv15-,,,-'1),1  ds  lf  J.o,  ,>0, 


Uf^d  -  e~Xt)  if  j  »  i  +  1. 


Since  the  system  is  a  Jackson  network,  the  steady  state  queue 
length  probabilities  embedded  at  inputs  is  known  to  be 


If  the  input  process  were  renewal  it  would  have  distribution 


r(t)  »  irQ(t)U  *>  1  -  e  Xt  -  ^  e  llt, 


and  Vx.y,  (n  -  ^^y)Q(y)U  “  °-  But 


_  l&*l)Q(y)l)  -  (Rl-  e“Xy  _  £iL  e'Wy  +  Pe-(u+X)y), 

r(x)  ^  7  r(x)  y—  A  p-A 


Since  r(x)  -  (1-e  M  )  <  0,  the  process  cannot  be  renewal  if  for  some  y 


BA—  e  _  £1L_  g-I'y  +  pe~^+^^  +  0.  Bv  the  linear  independence  of 
U-X  p-X 

exponentials,  this  expression  can  only  be  zero  if  p  =  0.  Thus 
if  there  is  a  positive  probability  of  a  customer  feeding  back,  the 
input  process  is  not  renewal.  If  p  *  0,  a  direct  calculation  shows  that 
Q(t)U  -  (1-e  Xt)U  (as  it  obviously  should  be).  By  theorem  2.2.2  the 
input  process  is  a  Poisson  process  in  that  case.  The  question  of  whether 
or  not  the  input  process  of  this  network  is  renewal  was  previously 


unresolved. 


Example  2.4.5.  Now  consider  the  M/G/l  queue  with  delayed  feed¬ 
back,  where  the  delay  mechanism  is  a  */M/l  queue. 


A  full  discussion  of  this  system  can  be  found  in  Foley  [7]. 

It  is  shown  that  if  the  departure  process  is  renewal  it  must  be  Poisson 
with  rate  A.  The  idea  of  equivalence  yields  an  interesting  corollary 
to  that  fact. 

Assume  the  G  server  has  service  time  distribution  g(t)  where  g 
is  differentiable  at  zero  and  g' (0)  =  p.  The  departure  process  is  most 
naturally  represented  by  a  MRP  with  state  space 

S  ■  (0,1,2,* x  {0,1,2,***}  since  the  future  of  the  system  is  condi¬ 
tionally  independent  of  the  past  at  a  departure  point  given  the  length 
of  both  queues. 

Let  y  *  “  0,1,2,***,  lexicographically  ordered}  be  a 

distribution  on  S.  Let  Q(t)  be  the  kernel  of  the  MRP  representing  the 
departure  process. 


Clearly  Q(t)  has  the  form 
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Q00(t) 

Sl"> 

Q02(t) 

Q03(t) 

.  .  . 

V£) 

QnM 

Ql2(t) 

Q13(t) 

.  .  . 

So"> 

QnM 

Q22(t) 

Q23(t) 

.  .  . 

So(t) 

Si"1 

Q32(t) 

Q33(t) 

.  .  . 

• 

• 

# 

• 

where  Q±  ^  C  t)  is  the  matrix  i',j'  e  {0,1,2,--*}* 

Although  Q(t)  cannot  be  easily  written  down,  it  is  a  simple 
matter  to  verify  that 


Q* (0)  = 


_ 

pql 

pql 

uqi 

• 

Say  (Q,y)  ~  r.  We  know  that  r(t)  -  1  -  e~Xt,  so  by  taking 
derivatives  the  definition  of  equivalence  yields 

Vn,  YQ'(0)nU=An. 


To  justify  the  last  step  note  that  Q' (0) 


Q(0)  =  0.  Thus 


lira  since 

t-K)  C 


VQ* (0) 


n.  ^  -  u.  r-w ; 

t-+0  f  t->0  c  c 


where  e  Likewise 


yQ' (0)n 


lim 

t  -*0 
n 


lim 

t  .->0 
n-1 


lim 

V0 


ru1)r(t2>  ■ 


r(tn> 


tlt2' 


t 

n 


”h*re  V,  r  e  J{.  Since  is  closed, 

C1  2  n  Y  Y 

YQ* (0)n  =  r' (0)n> 

where  y  c  so  yQ' (0)nU  =  r'(0)n. 

Let  y  =  (Y0.Y1.Y2,*  ")  where  Yj  =  (>j0,Yjl ,Yj2’ '  “  ) ■  By  the 
form  of  Q* (0)  we  have 

n  _ 

YQ’ (0)n  =  (0,0, "".O,  (pq)ny  ,  (pq)ny  ,••• ) 

n  n+1 

ao 

so  Yl^  ~  ‘  implies  Y  U  =  (~)  (1 — — )  .  The  conclusion  is 

i=n 

that  if  the  departure  process  is  renewal,  the  .marginal  queue  length  dis¬ 
tribution  at  the  G  server  must  be  geometric  with  parameter  — 

pq 

Say  the  G  server  has  a  distribution  g(t)  that  is  not  differenti¬ 
able  at  zero.  If  the  departure  process  is  renewal  we  must  have 

Y(Q(t)  -  Q(0))U  _  r (t)  -  r (0) 
t  t 

for  all  t  >  0,  and  since  r(t)  -  1  -  e”*1  we  must  have 

lim  xL<L(.t) q(0))u  .  x> 

t-*0  t 


If  g(t)  is  not  right-differentiable  at  zero  the  limit  does  not  exist  so 
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the  departure  process  cannot  be  renewal. 

Example  2.4.6.  Consider  an  M/M/l/N  queue  (N  >  0)  where  the 

arrival  rate  and  service  rate  depend  on  the  length  of  the  queue.  Let 

>  0  and  p^  >  0  be  the  arrival  and  service  rate  when  the  queue  length 
^  n  4"  1 

is  n.  Assume  lim  sup  -  <  1  -to  assure  a  steady  state.  Let  Q(t)  be 

n 

the  kernel  of  the  MRP  associated  with  the  departure  process.  If  N  <  ", 
Q  has  N  +  1  states  and 


Q*  (0) 


Assume  (Q,n)  ~  r  for  some  r,  where  n  Is  the  steady  state  vector. 

N 

irQ' (0)U  =  r'(0)  so  r'  (0)  =  ^  \  ^ p  ^ .  But,  for  n  >  N,  Q'  (0)n  =  0  so 

.1  =  1 

0  =  nQ'  (0)nU  =  r*  (())".  Thus,  r' (0)  =  0,  which  implies  tt ^  =  0, 

|  ■*  1,2,'**,N.  Since  Q  Is  Irreducible,  this  is  impossible  so  we 
conclude  that  the  departure  process  can  never  be  renewal. 

If  N  =  •»  the  situation  changes  slightly  since  (}'(0)n  is  never 


zero.  First  consider  the  case  where  Pj  =  p,  J  =  1,2,* •• 
(Q.y )  ~  r  implies  yQ* (0)nU  =  r'(0)n,  Vn,  which  implies 


In  this  case 


V  <T >  <>  -  T>- 

It  is  easily  verified  that  the  steady  stale  queue  length  distribution  Is 
it,  where 
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xoV  "x\ 

”  j  j  V 

'  yj 


This  Is  not  geometric  unless  Aj  =  A,  j  =  0,1,2,***.  Tims  the  departure 
process  cannot  be  renewal  unless  Aj  is  a  constant  (i.e.  the  M/M/1  queue). 
If  we  allow  pj  to  depend  on  j  the  equations  YQ'(0)nU  *  r’(0)n 


yield 


Tj  °’ 


Since  the  steady  state  queue  length  distribution  is 


Vi,,,xi-i 

H  ,  = - -  IT,.  , 

j  M1P2***pj  0 

it  is  again  impossible  to  have  a  renewal  departure  process  unless  A^  is 
a  constant  (i.e.  A^  =  r'(0)). 

We  cannot  deduce  from  these  equations  that  p^  must  also  be  con¬ 
stant.  In  fact,  by  [13,  theorem  3 1 ,  any  birth-death  queue  with  a  steady 
state  and  A  =  X  has  a  Poisson  departure  process  with  rate  X.  For 
example,  let  A  “  X  and  say 


(  j  U ,  J  =  0 , 1 , 2 ,  •  •  •  ,  N 

Np,  j  >  N. 


This  is  precisely  the  M/M/N  queue,  which  Is  known  to  have  a  Poisson 

departure  process  with  rate  X  when  In  steady  state. 

if  we  relax  our  assumption  that  the  p  's  and  X  's  be  nonzero 

r  n  n 

there  are  other  classes  of  queues  with  renewal  departures.  If  p^  -  0 
the  MRP  is  no  longer  irreducible  since  the  queue  length  can  never  be  less 
than  n  once  It  is  greater  than  n.  Likewise,  if  A^ 


0  the  queue  length 


17 


can  never  exceed  n  once  It  Is  below  n.  An  invariant  queue  length 

distribution  need  not  be  strictly  positive. 

bet  N  *.  Say  (l),y)  ~  r  for  some  y  and  r.  Let  n  -  lim  yi/'  (•") 

ii  *"■ 

be  an  invariant  distribution  that  satisfies  (Q,n)  r.  Say  N  0. 

It  Ay  s  t)  we  have  n  (  N  0  which  implies  |i(  -  l)  wliicli  implies  ni(  ~  0;  a 


0 


contradict  ion.  It  Ay  -  0  either  n  -  1,  in  which  case  there  is  no 

departure  process,  or  tiy  -  0. 

Sav  n  il  tor  some  1  "  0.  Then  vi ,  ■  0,  so  n,  -  0, 

1  .1  k 

k  «  0, l, 2, • • • , J  -  t.  If  A|  “  0  there  is  no  departure  process,  so  A^  N  0. 
It  we  also  had  A^  ■»  0  we  would  have  N  0  for  some  k  s  |  which  implies 

iij  *  0;  a  com  radical  ion. 

The  only  remaining  possibility  is  that  tor  some  J, 
n  -  (0,0, • • • l , • • *0) ,  p  -  0,  n  .  '0,  A  s  0,  A  -  0  (all  other  u’s 

1  l+l  1  1+1 

and  \’s  arbitrary).  In  this  case  the  queue  length  is  always  |  alter  a 

departure,  and  the  dpearture  process  is  renewal  with  distribution 

•t 


r(t) 


V 

y  1  (i 


o'V'1'  “’w.. 


Sav  N  -  •»  and  suppose  (i),n)  ~  r  where  n  is  an  Invariant  dlstrilui 

tlon.  Again,  if  p  *■  0  and  n  '0  then  n,  •  0,  k  «-  n.  Likewise  it 
n  n  k 

\(n  "  0  and  '  0  then  “0,  k  '  m  +  l.  The  arguments  given  tor 

N  <  >«’  show  that  the  departure  process  cannot  be  renewal  if  X  *  0, 

m 

n  >  0,  |i  “  0,  n  '  0  unless  n  •  m  +  1 . 
m  n  ii 

Thus,  the  only  other  wav  there  can  be  a  renewal  departure  process 

Is  as  follows.  Sav  p  -  0,  u  '  0,  A  „  ,  s  0,  k  -  1,2,* •■.  In  this 

n  nfk  n+k- 1 


ease 
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Q(t)  = 


A(t) 

B(t) 

0 

Q  (t) 

where  Q  is  irreducible.  In  fact  Q(t)  is  the  kernel  of  the  MRP  for  the 
departure  process  from  an  M/M/1  queue  with  =  ^n+k’  1  =  ''n+k  1’ 

k  *=  1,2,*  •*.  Let  n  be  the  steady  state  vector  for  Q.  If  (Q,ir)  ~  r  then 

A 

(Q ,  7t)  ~  r  where  ir  =  (0,ir). 


5.  Equivalence  as  a  Homomorphism. 

Let  (Q,y)  be  a  MRP  and  let  r(t)  =  yQ(t)U.  Associated  with  each 
t  e  I0,«]  is  a  matrix  Q(t).  Let  Q.  be  the  ]R-algebra  of  matrices  gen¬ 
erated  by  (Q(t)},  t  e  For  each  probability  vector  B  we  have  a 

map  F  :  Q  •*  K  where 

P 

Fg (A)  «  BAU. 

A  necessary  and  sufficient  condition  for  equivalence  can  now  be 
written  in  a  very  simple  form. 

Theorem  2.5.1.  (Q,y)  ~  r  if  and  only  if  F^  is  a  homomorphism. 

Proof.  («■*>)  If  (Q,y)  ~  r  then 

Vn.tj  t2,-**tn,  yQ(t1)Q(t2) • • •Q(tn)U  =  r(t1)r(t2)---r(tn). 

Also,  if  A^,  A2  e  Q  then  y(A^  +  A2)U  -  yAjU  +  yAjU  os  F^  is  a 
homomorphism. 

(♦■«)  If  F^  is  a  homomorphism  then  Vn, t^, t2> ■ • • t^, 
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YQ(t1)Q(t2)...Q(tn)U-  (YQ(t  )U)(YQ(t2)U)...(YQ(t)U)-r(t1)r(t0)...r(t  ), 

n  i  z  n 

so  (Q,y)  ~  r.  n 

The  maps  {Fg}  also  give  an  alternate  characterization  of  the  set 
J1^  when  Q  is  an  irreducible  MRP  with  steady  state  vector  n. 

Theorem  2.5.2.  8  e  if  and  only  if  is  a  homomorphism. 

(“■*“)  If  6  e  then  (Q,  8)  ~  r  where  r(t)  -  8Q(t)U  so 

Fg  is  a  homomorphism  by  theorem  2.5.1. 


,  x  co  BQ(x  )Q(x  ).-.Q(xm) 

<«— )  Let  <?  -  {y:  y  -  -y— - 

r(x. )r(x,) • • • r(x  ) 

If  Y  e  9P  then  F  is  a  homomorphism  since 


for  some  x^,  x2,‘' 


,x  }. 
m 


YQ(t 


3Q(x.)Q(x,) > • *Q(x  ) 
_ jl _ £■ _  m 


1)Q(t2)"'Q(tn)U  “  Q(t1)Q(t2)-.-Q(tn)ll 

r(x1)r(x2)*..r(xn)r(t1)r(t2)**T(tn) 


r(x1)r(x2)-  •  rr(xJ 


“  r(t.  )r(t.) • • • r(t  ), 
1  t  n 


Furthermore,  F^  -  Fg.  Clearly  if  Y  t  $  ,  F^  =  F?  and  if  Y  is  any 

then  F  *  F  .  Thus  F 
Y  p  if 


linear  combination  of  elements  in 
so 


8  e  CK  .  CJ 

7T 


V 


6.  Collapaibility. 

Let  (q.if)  be  a  k  state  MRP  (k  <  »)  and  let  {A.  ,A, ,  •  •  ■  ,A  },  (m<k) 

i  l  m 

be  a  partition  of  the  states  of  (Q,n).  Let  n,  U  be  as  defined  by  (1.2.1) 
and  (1.2.2). 

nQ(t)U  is  an  m  x  m  matrix  whose  (i,j)  element  is 

P*<Xi  e  Aj»  Tj  1  1  I  Xq  e  Ai^’  Say  F  is  tI,e  function  that  maps  the  state 

space  of  (Q.it)  to  (A  ,A  ,  •  •  •  ,A  } .  Serfozo  [16]  shows  that  if  {F(X  ),T  ) 

1  L  m  n  n 

is  a  MRP,  its  kernel  is  TlQ(t)n. 


i 
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The  definition  of  equivalence  between  a  MRP  and  a  renewal  process 
has  a  natural  generalization  to  collapsibility  between  two  MRP's. 

Definition  2.6.1.  Let  (Q » tt )  be  a  MRP  with  state  space  {l,2,***,k} 
(k  <  •»)  and  let  Y  be  an  ro  state  MRP  (m  <  k).  Let 

F:  {l,2,***,k}  -*  {A,  ,A.,***,A  }  be  a  partition  of  the  states  of  (Q,n). 

I  z  m 

F 

We  say  (Q,ir)  is  collapsible  to  Y  via  the  partition  F,  written  (Q,n)  ~  Y, 
if 


Vn,tlft2.*.*,tn,  nQ(t1)Q(t2)- * -Q(tn)U  -  Y(t1)Y(t2)-..Y(tn). 

If  F  is  clear  from  the  context  we  will  write  (Q,tt)  ~  Y.  Note  that  the 
initial  distribution  of  Y  depends  only  on  it  and  F;  namely  y  =  itJU. 

F 

Lemma  2.6.1.  If  irQ(<*>)  «  tt  and  (Q,u)  ~  Y  then  yY(“>)  ■  y  where 

y  *  irll. 

*  -v 

Proof.  Since 


keF  *(i) 

we  have  yY(<»)  -  y jJ  -  ttUIIQ(«)P  -  irQ(«>)n  -  nM  =  y.D 

In  the  case  where  Y  has  one  state  (a  renewal  process)  we  showed 

that  collapsibility  is  the  same  as  weak  lumpability.  If  (Q,ff)  is 

{X  ,  T  }  and  (Y,y)  is  {Z  ,  S  }  (where  y  -  nU) ,  then  definition  2.6.1  says 
n  n  n  n 

that  for  each  i,  j  e  (1,2, ••*,«}  and  Vn, t^ , t2« • • • , t  , 


if  j  t  F_1(i) 
if  j  e  F_ 1 ( i ) , 


(2.6.1)  PTt(XneAj,Tn<V---,T1<t1|X0EAi). 

For  weak  lumpability  between  (X^, 
that  for  each  i,  j  c  {l,2,***,m}  and  Vn,t 


■  P(Z  -j,S 
n  J  n 

T  }  and  {Z 
n 

»■  •  •  •  f- 

l,c2*  ’ 


<  t 


n* 

n' 


S  }  we  would  need 
n 

and 


ilti2, e  U,2, • • -.m}, 


fc 


I 


Lm _ I _ 


i 
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(2.6.2)  VVV*«-1EA1  •  •  •VV-  T„±  V"’1!  i  ‘dW 

n-1  1 

■  'V'Vl'Vl'-'2!’1!'  VV'-V'llV0- 

It  seems  inconceivable  that  every  {X  ,  T  }  and  {Z  ,  S  }  that 

n  n  n  n 

satisfy  (2.6.1)  would  also  satisfy  (2.6.2)  but  all  attempts  to  find  a 
counterexample  have  failed  so  far.  Clearly  weak  lumpability  implies 
collapsibility,  though. 

Since  the  definition  of  collapsibility  between  two  MRP’s  is 
analogous  to  the  definition  of  equivalence  between  a  MRP  and  a  renewal 
process  one  might  suspect  that  the  conditions  for  collapsibility  would 
be  similar.  First  of  all,  any  sufficient  condition  for  weak  lumpability 
will  also  be  a  sufficient  condition  for  collapsibility.  Thus  we  ha\<? 

Theorem  2.6.2.  If  IIQ(t)  *  Y(t)n,  Vt  then  (Q.y)  ~  Y  for  any  y 


that  satisfies 


jeA,  J 


II..  whenever  k  e  A . . 

ik  i 


Theorem  2.6.3.  If  Q(t)U  -  JfTY(t:)  ,  Vt  then  (Q.y)  ~  Y  for  any  y. 
Theorems  2.6.2  and  2.6.3  can  be  proved  the  same  way  theorems  2.2.1  and 
2.2.2  were  proved.  They  can  also  be  found  in  Serfozo  [15]. 

Let  (Q,n)  be  a  k  state  MRP,  let  Y  be  an  m  state  MRP  and  let  F  be 
a  map  from  the  state  space  of  (Q,tt)  to  the  state  space  of  Y.  l.e t » bo 
the  set  of  all  m  x  k  matrices,  M,  with  M^  >  0  and  My,  ■  nl  where  11  is 
the  summing  matrix  induced  by  F  and  a  is  a  scalar.  In  other  words,  M 
must  have  the  form 


M 


X-*  -X 

X*  •  •  x 

.....  .  _ 

J 


and  each  row  sun  must  be  the  same.  Let  «fF-  MQ(t)J^*  (MU)Y(t),  Vt) 

and  let  be  the  largest  subset  of  that  satisfies 

^(t)CV(t)  Vr  Vt.  Let  *  {Me  3^p :  MU  -  1}.  These  sets  are 

analogous  to  the  sets  defined  in  section  2  and  not  too  surprisingly  we 
have 

Theorem  2.6.4.  (Q,n)  ~  Y  if  and  only  if  n  c  Jjfp. 

Proof.  (-^)  If  n  then  n  t  )fp.  Thus,  for  some 

*1’ t2* ' " * • tn*  NQ(t^)Q(t2) *  * *Q(tn)  i  but  this  says  that  for  some  t 

bq(t1)Q(t2)-*-Q(tn)q(t)u  i  nq(t1)q(t2)---q(tn)UY(t) 

-  Y(t1)Y(t2)***Y(tn)Y(t). 

F 

This  contradicts  (q,ir)  ~  Y. 

<-)  IE  n  e  J1TF  then  Vn, t j , t2> • • • , tn. 

nq(tl)q(t2).*-q(tn)  c  Yft^Yft,) •  •  -Y(tn>  ,Xf 

so 

nq(tl)q(t2)---q(tn)U  -  Y(t1)Y(t2)---Y(tn).n 

Another  result  that  carries  over  easily  from  the  renewal  case  is 
that  collapsibility  is  identical  to  a  certain  algebra  homomorphism.  Let 
Q.  be  the  ring  generated  by  {q(t)},  t  c  JO,®)  and  let  V  be  the  ring 
generated  by  (Y(t)},  t  c  [0,“J.  Let  ♦  Q.  -*■  V  be  >1  _(A)  -  I1AU. 

Tr ,  r  n ,  r 

p 

Theorem  2.6.5.  (q.n)  ~  Y  if  and  only  if  „  Is  a  homomorphism. 

-  n ,  r 

The  proof  here  is  identical  to  the  proof  in  the  renewal  case. 

One  result  that  has  no  counterpart  in  the  renewal  case  is 

Theorem  2.6.6.  Let  (q,x),  Y  and  D  be  k,  m  and  t  state  MRP's 

F  C  (>F 

(k  >  m  >  t).  If  (q,n)  ~  Y  and  (Y,y)  ~  D  (where  y  -  nlO  then  (q.x)  ~  D 

Proof .  Let  H  and  T  be  the  matrices  induced  by  the  vectors  «  and 

y  and  the  partitions  F  and  G.  Let  ‘n'  be  the  matrix  induced  by  *  and  the 

partition  G»F  .  First  of  all,  a  direct  calculation  yields  HI  -  n\  Also 
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~P^G  "  -^of  where  Up*  and  JiGc>p  are  the  summing  matrices  associated 

with  the  partitions  F,  G  and  G«F.  Since  (Q,n)  ~  Y,  Vn, t, , t,, • • • , t  , 

1  i  n 

nQ(t1)Q(t2)---Q(tn)iJF  -  Y(t1)Y(t2)*"Y(tn).  Also,  since  (Y,y)  ~  D, 
rY(t1)Y(t2).*.Y(tn)^c  -  D(t1)D(t2)-*-D(tn).  Thus 

rnQ(t1)Q(t2)---Q(tnUfF4!G  “  D(t1)D(t2)**-D(tn).  Thus 

/n^(t1)Q(t2)---Q(tn)i!GoF  «  D(t1)D(t2)---D(tn),  so  (Q.n)  G~F  D.  g 

7.  Equivalence  Between  two  MRP’s. 

In  section  6,  equivalence  between  a  MRP  and  a  renewal  process 
was  generalized  to  collapsibility  between  two  MRP’s.  The  results  in 
that  section  will  prove  to  be  useful  in  the  next  two  chapters.  In  view 
of  (2.6.1)  and  (2.6.2),  though,  it  seems  that  the  real  meaning  of 

p 

(Q.y)  ~  Y  is  cloudy.  A  more  intuitively  appealing  generalization  of 
equivalence  between  a  MRP  and  a  renewal  process  is  given  by 

Definition  2.7.1.  Let  (Q,y)  and  (Y,S)  be  two  MRP’s.  We  say 
(Q.y)  is  equivalent  to  (Y,6)  (written  (Q,y)  ~  (Y,£))  if 

Vn,ti,t2,  *  •  •  ,tn  YQ(t1)Q(t2^  " *  3Y(t1)Y(t2)  *  *  •  Y(tn)U. 

Let  (Q.y)  be  Un,  Tn>  and  let  (Y.B)  be  {Zn>  Sn}.  If  (Q,y)~  (Y.tO 
then  the  distribution  of  {T^,T2»*"}  is  the  same  as  the  distribution  of 
{S^,S2,"*}.  Strong  lumpability,  weak  lumpability  and  collapsibility 
are  not  really  equivalence  relations  between  MRP’s  since  the  relation  is 
always  from  the  bigger  one  to  the  smaller  one.  Equivalence  is  an 
equivalence  relation  between  MRP’s.  It  is  also  the  condition  that  must 
be  satisfied  by  two  random  processes  if  one  is  to  be  substituted  for  the 
other  in  (say)  a  queueing  system.  Furthermore,  it  is  a  weaker  condition 
than  strong  lumpability,  weak  lumpability  and  collapsibility. 
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Theorem  2.7.1.  Let  n  and  jJ  be  given  by  (1.2.1)  and  (1.2.2).  If 
(Q,ir)  ~  Y  then  (Q,ir)  ~  (Y,d)  where  $  *  irj£. 

Proof .  (Q,e)  ~  Y  implies 

Vn,t1,t2,---,tn,  nQ(t1)Q(t2)--*Q(tn)U  -  Y(t1)Y(t2)--*Y(tn). 

If  8  ■  wJJ  then  6n  «  irJJll  «  *,  so 

*Kt.)q(t,)*--Q(t  )u  -  dnq(t.)Q(t,)---Q(t  )uu 

i  ±  n  l  4.  n  ^ 

-  8Y(t.)Y(t,)---Y(t  )U. 

i.  4-  II 


Thus,  (Q.w)  -  (Y,6).  n 


CHAPTER  111 


EQUIVALENCE  BETWEEN  CANONICAL  MRP'S 


1.  The  Core 

Theorem  2.2.7  states  that  (Q,y)  ~  r  if  and  only  if  y  Cjl\  where 
•*,  is  the  largest  set  of  probability  vectors  satisfying 

Jif'Q(t)  Cr(t>  t*  .  Vt  and  SQ(t)U  -  yQ(t)U,  V*  e  ,Xy- 

By  Corollary  2.2.5. 1,  if  Q  is  irreducible  and  (Q,y)  ~  r  then 

(Q,it)  ~  r  where  n  is  the  steady  state  vector  for  Q.  Thus,  if  (Q.y)  ~  r 

and  (Q.B)  ~  r  then  Jjf  *  and  'X  -  ,X.  so  ”  tX.-  111  is  leads 

y  *  b  it  y  b 

to  the  following  definition. 

llefinition  3.1.1.  Let  Q  be  an  irreducible  MRP  with  steady  state 
vector  it.  The  core  of  Q  is  the  set  ,X  •  It  will  be  denoted  t) i. 

IT 

If  'X  “  then  Q  is  not  equivalent  to  any  renewal  process.  For 

any  probability  vector  y,  either  'X  -  ♦  or  tX  - 

y  y 

Theorem  3.1.1.  If  Q  is  irreducible  with  steady  state  n  and 
Q  ~  r  then  r(t)  -  TtQ(t)U. 

*#•%  . 

Proof .  Q  ~  r  implies  that  for  some  y,  (Q.y)  ~  r.  Since  Q  is 
irreducible  y  e  <X  so  yQ(t)U  -  irQ(t)U,  Vt .  Thus  r(t)  -  irQ(t)li.U 

In  general  it  is  very  difficult  to  determine  what  *X  is  unless 
tt  can  be  shown  to  be  empty  or  contain  a  single  vector.  Clearly,  if  *X 

consists  of  a  single  vector,  it  must  be  the  steady  state  vector.  If 
Q  ~r  where  Q  is  a  finite  state,  irreducible  MRP,  one  can  find  a  set 
that  must  be  Inside  the  core. 
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Theorem  3.1.2.  Let  Q  be  a  finite  and  irreducible  and  assume 
Q  ~  r.  Let  [0,"]  be  the  set  (t:  Q(t)  is  irreducible}.  For 

each  t  e  let  y  be  the  unique  probability  vector  that  satisfies 
YtQ(t)  *  r(t)yt  and  let  be  the  linear  subspace  generated  by  {Yt}> 
t  e  3".  Tn.e.'n.fyif 

Proof .  By  Lemma  2.2.5,  w  is  convex,  compact  and  invariant  under 
multiplication  by  Q(t)/r(t).  Thus  there  exists  yt  cJtf^that  satisfies 
YtQ(t)/r(t)  *  Yt  by  the  Brouwer  Fixed  Point  Theorem.  Thus,  if  6  e^ 
then  BQ(t)U  =  (3U)r(t)  since  6  is  a  linear  combination  of  elements  of 
and  3Q(t)/r(t)  eW  since  it  too  is  a  linear  combination  of  elements  of 
CfC.  Thus  (<^fl^)Q(t)/r(t)  C  (^TV^)  which  implies  that  <WT1 
Example  3.1.1.  We  continue  example  2.2.1.  Recall 
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f  0  if  t  <  t* 


,1  1  lx  ..111. 

'3*  3*  3) »  ^  —  \^i  2*  2^*  t(t)  * 


1  .  .  *  ** 
2  if  t  <  t  <  t 


if  t  >  t 


** 


"fc  <^cik 

Note  that  yQ(t)  =  r(t)y  if  t  _<  t  <  t  and  it  is  the  steady  state  vector. 
Since  Q  ~  r  we  know  that  both  y  and  it  must  be  in  (this  was  also  proven 
in  example  2.2.1). 

Consider  the  vector  B  =  (1,0,0),  which  is  not  a  linear  combination 
of  y  and  tt.  Since  BQ(t)U  =  when  t  lc<t  ,  6  t  CH,  ■  By  theorem 
3.1.2,  the  intersection  of  the  line  through  y  and  tt  and  is  in  £f{  . 
Since  B  i  £t{  ,  Cfi  cannot  be  two  dimensional.  Thus  £f{ is  the  line 
through  y  and  tt  on  . 


If  Q  is  not  irreducible  there  is  no  unique  steady  state  vector 
so  one  must  be  careful  when  talking  about  equivalence  and  about  the  core. 

2.  Reducible  MRP’s 

The  main  problem  with  reducible  MRP's  is  that  they  can  be 
equivalent  to  different  renewal  processes  depending  on  the  initial 
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distribution.  For  example,  say  (Q^r^)  ~  r  ami  ((^.i^)  ~  r2*  If 
n  -  (tt^O),  it*  =  (O,^)  and 

Q(t)  « 

then  (Q,ir)  ~  r^  and  (Q,it')  ~  r2- 

Another  problem  with  reducible  MRP's  is  that  in  general  they 
contain  some  transient  components.  Recall  the  canonical  form  of  a  kernel 

(1.3.1) .  The  states  corresponding  to  the  A^(t)'s  are  irreducible  sets 
of  recurrent  states  and  the  states  corresponding  to  the  B.(t)'s  are 
irreducible  sets  of  transient  states.  Let  rjT  be  the  set  of  states 
corresponding  to  the  B^'s.  Let  Q(t)  have  the  form  (1.3.1). 

Theorem  3.2.1.  If  (Q,y)  ~  r  then  there  is  a  probability  vector 
y'  such  that  y!  =  0,  Vj  e  ^  and  (Q,y')  ~  r. 

Proof .  Globally,  Q(“)  has  the  form 

(3.2.1) 

where  A  and  B  are  square  matrices,  and  Qn(°°)  has  the  form 


An 

Cn 

Bn 

Since  B  Is  a  transient  MRP,  11  >  0  pointwlse  anil  11m  C  Is  well 

n 

defined.  Thus 

Itm  (yQ"(“)) .  -  0.  Vj  ,  if. 

n+» 

By  theorem  2.2.7,  (Q.y)  ~  r  Implies  y  i  rXy-  Since  '  Is 
closed,  convex  and  invariant  under  multiplication  bv  t)(‘”)  we  have 

n-Hit- 1 

*  1  X  ^  L 

Y  “  llm  in  Y‘}  ^  '  '*v 

tv***  f— '  ' 

k“n 

where  m  Is  the  period  of  the  states  in  A  (l.e.  the  least  common  multiple 

*  *  U 

ol  the  periods  ol  the  Aj  s) .  y  satisfies  y^  ”0,  »|  t  ,/  and 

* 

(Q.  y  )  -  r.  i  i 

Corollary  1.2. 1.1.  bet  Q  have  the  lorm  1.1.1  where  A  Is  k  x  k. 

II  (ti.y)  '  r  then  (Q.y')  -  r  where  y'  Is  made  up  ol  the  I  list  k  entries 

* 

Ol  y  . 

* 

Proof.  From  the  theorem,  (t},y)  ~  r  Implies  (Q,Y  1  "•  r  where 
Y|  -  0,  i  i/t  .  Thus, 

Vn.  t  . ,  I  t  ,  y*i}(  t  i  >Q(  t  >V  *  *Q(t  ,)!'  -  r(t  lr(t  .,)•••  r(t  )  . 

I  11  I  A  II  I  »  II 

But  y  iMtjWd  ,)••  •t,Htn)U  ~  y'A(t  )Mt.,)‘*-A(tn)U.  so  (Q. y')  -  r.  r 

In  essence,  we  have  shown  that  as  tar  as  equivalence  to  a  renewal 
process  Is  concerned,  transient  components  are  irrelevant.  We  will 
return  to  MRP's  with  transient  components  in  a  dlllerent  context  in 
section  "i. 


A  general  MRP  without  transient  components  has  a  kernel  ol  the 


5 


0 


(3.2.2)  Q(t) 


■ 

■ 

Q2(t) 

• 

Q(t) 

m 

where  («>)  is  an  irreducible  x  stochastic  matrix. 

Theorem  3.2.2.  Let  y  *  (p^,  P2Y2’ "  *  *  ,PmYm^  where  Yj  has  nj 
elements,  y^l)  -  1,  Vj.  and  *  1*  Let  p  =  ^pi,p2’ 


■,P  )  and 
rm 


Y(t) 


rL(t) 

r2(t) 

4 

« 

_ _ 

r  (t) 
m 

Then  (Q.y)  ~  Y  if  and  only  if  (Q^.y^)  ~  for  each  j. 

Proof.  (*■»  )  If  (.Qj , Y j )  ~  Tj  then  the  core,  £f{ ^  of  contains 

V  Ut 
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Since  r  e  Jtf,  J$fQ(t)  C  Y( t)J{  ,  Vt  and  Y(t)  =  TQ(t)U,  we  have  (Q,y)  ~  Y. 

Note  that  the  starting  vector  for  Y  is  yU  =  p. 

(==*►)  First,  Pj  >  0,  Vj  since  if  not  r^(t)  “  0*  Vt.  Thus, 

since  (Q,y)  ~  Y  there  must  be  an  invariant  set  of  matrices  CK  with 

r  e  Cfi  such  that  J^Q(t)  C  Y(t)Jflf  ,  Vt.  Let  Jfcf.  be  the  set  of  vectors 
—  ~  ~  j 

that  correspond  to  the  j1"*'  row  of  the  matrices  in  M.  Thus 

Qj  (t )  G  Tj(t  )ty{  y  Vt.  Since  y^  e  ,  this  implies  (Qj»Yj)  ~  r^.M 

The  main  result  of  this  section  says  that  if  a  reducible  MRP 

is  equivalent  to  a  renewal  process,  then  every  component  (with  positive 

initial  probability)  must  be  equivalent  to  that  same  renewal  process. 

Theorem  3.2.3.  Let  Q(t)  have  the  form  (3.2.2)  and  let  be 

the  unique  vector  satisfying  w,Q.(")  =  n.,  u.U  -  1.  Let 

J  J  m  J  J 

tt  =  (p,n, ,  •  ,p  tt  )  where  V*  p.  *  1.  Then  (Q,tt)  ~  r  if  and  only 

1  i  2  2  mm  <  1 

J=1  ' 

if  for  each  j  either  (Qj.Hj)  ~  r  or  Pj  *  0. 

Proof .  (•*=  )  Let  tfle  set  pj  >  Clearly» 

(Q,7t)  ~  Y  where 


Q,  (t) 
J1 

1 

i 

1 

Q,  (t) 
J2 

Y(t)  - 

• 

Q,  (t) 
Jk 

,  JK 

~%r\ 

IX 

By  theorem  3.2.2,  (Y ,  w * )  ~  Z  where  it' 


(p,  ",  »P,  ",  »•••,?,  ",  )  and 
J1  J1  J2  J2  Jk  3k 


Z(t) 


r(t) 

r(t) 

• 

r<t) 

Since  Z  is  strongly  lumpable  to  r  we  have  (Z,  p')  ~  r  where 

P'  ■  (P,  .P,  )•  Thus,  (Q,ir)  ~  Y,  (Y,tt')  ~  Z  and  (Z,p')  ~  r. 


■*r*2 


j 


By  theorem  2.7 .6  we  have  (Q,n)  r. 

(  — *)  If  (Q,ir)  ~  r  then  Vn,  t^tj,  ■  *  *  ,tn, 

"Q(t1)Q(t2>- • *Q(tn)U  -  r(t1)r(t2) • • -r(tn> .  By  the  form  of  Q(t), 

TrQ(tl)Q(t2)--.Q(tn)U  -  s 

For  any  N, 

"Q(t1)Q(t2)-  ••Q(tn)QN(“>)Q(t1)Q(t2)”-Q(tn)U  -  (r(t1)r(t2)-  •  -r(tn))2. 

Thus,  for  any  N, 
m 

^j"jQJ  (tl)Qj  (C2)  *  *  *Qj  (tn)Qj  (°°)Qj  (tl)Qj  (t2)  "  'Qj  (tn)U 

-  (r(t1)r(t2)--»r(tn))‘ 

Say  Q  (<»)  is  periodic  with  period  k,.  We  have  for  any  N, 

m  3  r 


tf 


(r(t1)r(t2)---r(tn))‘ 


Thus, 
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H+kj-1 

£pJ.JQJ(tl)Q1(t2)--.g),tn)r.i.  tjr  q;<-)]<ij<«1)qj 

j-l  L  J  l-N  J 


<e2>---Qj(tn)u 


(r(t1)r(t2>  •  •  •  r(tn>)' 


Let  nj^J  *  ajYj  wllere  Since 


N+k.j-1 


NT*,  a- X 

L  i-N  J 

^PJ1TjQj(tl)Qj(t2)“'Qj(ti 


we  have 


N+k i-l 


•Ev? 


Thus , 


^^Pjaj  "  (r(t1)r(t2>- •  *r(tn))2, 

ID 

^Pjaj  ’  r(t1)r(t2)--.r(tn), 

t V' '  ®VJ]2 '  ° 


This  term  says  that  the  variance  of  some  random  variables  that 
take  values  witli  probability  Pj  is  zero.  The  only  way  this  can  be 
is  if  there  is  some  common  value,  a,  such  that  for  each  j  either  a^  ■  a 


or  Pj  *  0. 


Thus  r(t1)r(t9)***r(t 


»' '  £pj“j 


a.  This  says  that  if  p  >  0 


then  njQj(t1)Qj(t2)“*Qj(tn)U  -  r(tj)r(t2) •  •  •  r(tn> ,  Since  n, tj  ,  1 2>  •  •  • .  t n 
were  arbitrary,  we  have  (Qj ,  it  j )  ~  r.  n 
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Theorem  3.2.3  is  not  the  strongest  result  imaginable  on  this 

topic.  A  stronger  result  would  be 

Conjecture  3.2.1.  If  Q  has  form  (3.2.2)  them  (Q,y)  ~  r 

(where  y  *  (p.y, ,  PoY-»»***»P  Y  ))  if  and  only  if  for  each  j  either 
l  l  l  t  n  n 

(Qj.Yj)  ~  r  or  pj  -  0. 

Although  ~  r  or  ■  0  for  all  j  clearly  implies 

(Q.Y)  ~  r,  the  idea  behind  the  proof  of  theorem  3.2.3  cannot  be  used  to 
prove  the  converse.  The  problem  is  that  in  general  we  cannot  force  the 
distribution  on  the  state  space  back  to  y  once  it  leaves  it.  We  can 
force  it  back  to  n  by  looking  far  enough  into  the  future.  We  can  say 
something  slightly  stronger  than  theorem  3.2.3  though. 

Corollary  3. 2. 3.1.  If  (Q,y)  ~  r  then  for  each  j  either 
(Qyi^)  ~  r  or  p^  -  0. 

Proof.  Since  (Q,y)  ~  r,  (Q,x)  ~  r  where  it  «  lim  y 

n->°° 

and  m  is  the  period  of  Q.  The  result  follows  from  the  theorem,  n 

The  theorems  in  this  section  can  be  reformulated  in  an  enlighten¬ 
ing  way  if  we  extend  the  notion  of  a  core  to  reducible  MRP’s. 

Definition  3.2.1.  A  set  will  be  called  a  core  of  a  MRP,  Q, 
if  there  is  some  renewal  process  r  such  that  J^Q(t)C  r(t)J^,  Vt,  andjjf 
is  the  largest  set  with  that  property. 

If  Q  is  irreducible  the  core  is  unique.  If  Q  is  not  irreducible 
there  may  be  several  cores.  For  example,  say 
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where  Q^(t)  and  (^(t)  are  irreducible  and  there  exist  cores  J*i  and  CK2 
such  that  JST1Q1(t)C  r^t)  and  J^Q^t)  C  r2(t)  CHV  Vt  (i.e. 

Qj_  ***  and  Q2  ~  r^)-  Then  (  <25^, 0)  and  (0,  are  both  cores  of  Q 

yielding  different  renewal  processes. 

Theorem  3.2.4.  If  and  are  different  cores  of  Q  then 

1 

Pro°f •  Say  Q(t)U  -  r^(t)  and  “  ^(t)  where  r^r,. 

Let  y  -  <PiYrP2Y2*--*.PnYn)  and  B  -  c 

Say  (Pj^j) (®j* Yj)  1*  0  for  some  j.  By  theorem  3.2.3  this  means 
(Qj » j )  ~  r^  and  (Q^.n^)  ~  r2  which  is  impossible.  Thus  either  PjPj  “  0 
or  q^Yj  ■  0  (or  both),  so  y  •  S  *  0.  n 
Let 


Q(t) 


Q^t) 

_ 

— 

q2(t) 

* 

Qn(t) 

and  suppose  kH.  is  a  core  of  Q  yielding  the  renewal  process  r.  Let 
J  C  {1,2, ••♦,n}  be  the  set  of  indices  such  that  there  exists  some 
Y  e  CK  with  PjY j  i  0  (y  ■  (p^Y^>  P2Y2’ '  ”  ,PnYn^ '  By  corollary  3.2. 3.1, 
Qj  ~  r,  Vj  e  J.  Let  -  (0,0,  ••*,(),  Jj(  ,0,0, . .  •  .0)  where  JJf 

(the  core  of  Q^)  is  in  the  j*1*1  spot.  Let  $  be  the  convex  hull 


of 


u  «&. 


In  other  words  is  all  vectors  of  the  form 


JeJ 
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n 

*PlYl’p2Y2,'**’pnYn)  where  "  l*  pj  "  0  if  J  *  J*  and  Yj  e  Jtfj  lf 

J  c  J.  Clearly  icjf. 

/N 

Conjecture  3.2.2. 

Theorem  3.2.5.  Conjecture  3.2.2  Is  true  if  and  only  if  conjec¬ 


ture  3.2.1  is  true. 

Proof.  ('—■«)  Say  tSl  “  and  (Q,y)  ~  r  where 

y  ■  ^P1-'ri,P2Y2’*"*,PnYn^‘  Then  Y  “  ^])pjYj  where  Yj  f  t#.j  •  Thus 

jcJ 

(Q  ,Yj)  ~  r  whenever  t4  0. 


(♦— )  If  (Q,y)  ~  r  implies  (Q^ , y ^ >  ~  r  whenever  p^ 
V  *i  whenever  p^  i4  0.  Thus  any  y  is  of  the  form 
where  r  .  o 


j4  0  then 

D* 

jfJ 


3 .  Periodic  MRP's 

In  this  section  we  consider  MRP's  whose  states  are  arranged  so 
that  the  kernel  has  the  form 


1  <  n^  <  •,  J  ■  1,2, • • • ,m,  and  (t)  is  irreducible. 


Such  a  MRP  is  said  to  have  period  m. 


Since  a  MRP  of  the  form  (3.3.1)  is  irreducible,  by  theorem 
3.1.1  there  is  a  unique  core  CK  ,  so  if  (Q,y)  ~  r  then  r(t)  ■  irQ(t)U 
where  it  is  the  steady  state  vector. 

Let  Q(t)  have  the  form  (3.3.1)  and  let  it  be  the  steady  state 
vector  for  Q.  Let  it  -  ^(lyr^,  •  •  •  where  has  n^  elements  and 

n^U  »■  1.  It  might  seem  possible  for  Q  ~  r  even  if  (t)U  depends  on 
j.  Since  one  cannot  be  sure  which  phase  the  process  starts  in,  r(t) 
could  be  an  average  of  the  random  processes  arising  from  each  phase.  It 
turns  out  this  cannot  occur. 

Theorem  3.3.1.  If  Q  ~  r  then  n^(t)U  =»  r(t),  Vj- 
Proof.  If  Q  ~  r  then  Vn, 

irQ(t1)Q(t2)- • -Q(tfi)U  -  r(t1)r(t2)-*-r(tn).  In  particular, 
wQm"1(“)Q(t1)Qm“1(“)Q(t2)-”QD,_1(»)Q(tn)U  -  r(t1)r(t2)---r(tn).  Thus 
(Q,ir)  ~  r  where  Q(t)  «  Qm  *(”)Q(t).  But 


Q(t)  - 


Qx(t) 

Q2(t) 

• 

_ 

where  Q^(t)  -  Qj  (»)QJ+1(®)  •  ■  •Qjn(»)Q1(“)  •  •  • 

By  theorem  3.2.3  we  must  have  CQj  * w j  ^  ~  r»  V  j*  Thus 
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wjQj <t)U  ■  r(t),  Vj.  Since  we  have 

it^Qj(t)U  -  w^_iQj_1<t)U,  so  TT^Qj(t)U  -  r (t) ,  Vj.  a 

The  following  results  will  be  useful  in  the  next  chapter. 

Theorem  3.3.2.  Let  Q ( t )  have  the  form  (3.3.1)  and  Y(t)  have  the 

form 


Y(t)  - 


^(t) 

• 

• 

r  1<C> 
m-l 

rm(t) 

m 

Let  ir  »  —  (it,  ,n-,  •  •  *n  )  be  the  steady  state  vector  for  Q.  If 
ml/  m 

ifjQj(t)  m  rj  (t)nj+l*  then  ~  Y* 

Proof .  We  have  IIQ(t)  ■  Y(t)Il  where 


jEU.  . 


[.  .V 


so  the  result  follows  from  theorem  2.7.2.  o 

Theorem  3.3.3.  (Q,y)  ~  Y  if  and  only  if  there  are  sets 

n2  nm 

*y(  i » <y( 2  *  *  *  *  •  of  probability  vectors  in  ]R  ,  ]R  ,•••,* 
respectively  such  that  J^Qj(t)C  rj  (t)  j  “  1,2, 

yj  cMi- 


V  t ,  and 


Since  (t)  C  (t)  «#j+1*  V  j  we  have  JtfQ(t)  C  Y(t),^  so 

(Q.Y) ~  Y. 

( •^■i  it)  Since  (Q,y)  ~  Y  there  is  a  set  of  matrices  satisfying 
^fQ(t)  C  ^ »  V t •  Let  *  be  the  set  of  vectors  corresponding  to 
the  row  of  tX  .  Let 


which  is  in  ,X .  Thus, 

<v 


6 


ffi 


TQ(t) 


YjQ^t) 


Pm-lSn-l^ 


Since  J^Q(t)  C  Y(t)tXf  there  is  a  f  e 


r' 


Y ' 
m 


such  that  TQ(t)  *  Y(t)r'.  But 


Y(t)r'  - 


1 

rl(t)y* 

• 

• 

r  .(t)Y' 
m-1  ra 

r(t)Y: 
m  1 

J 

Thus  for  any  Yj  e  there  is  a  yj+1  t  J^+1  such  that 
YjQj(t)  -  rj  (t>Yj+1*  This  implies  J^Qj  (t)  C  ^  (t)  J^j+1>  VJ, 


4.  MRP’s  with  transient  states. 
Let 


(3.4.1) 


Q(t)  - 


A(t) 

0 

C(t) 

B(t) 

Vt. 
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where  A(t)  and  B(t)  are  square  and  Irreducible  and  C(t)  ^  0.  In  section 
2  it  was  shown  that  Q  ~  r  if  and  only  if  A  ~  r  so  in  the  context  of  that 
section,  the  transient  components  could  be  ignored.  If  we  examine  the 
situation  a  little  more  closely  it  becomes  apparent  that  MRP's  with 
transient  components  can  exibit  some  interesting  properties  of  their  own. 

Let  (Q,y)  be  a  MRP  where  Q(t)  has  the  form  (3.4.1).  Let 
Y  *  ^Y^.Y^  where  y^  corresponds  to  A  (the  recurrent  components)  and  Y2 
corresponds  to  B  (the  transient  components).  Clearly  (B,Y2)  is  a  tran¬ 
sient  MRP.  When  equivalence  between  a  MRP  and  a  renewal  process  was  de¬ 
fined  there  was  nothing  preventing  the  MRP  from  being  transient.  Thus, 

(B,y2)  ~  r  if 

Vn,t1,t2> • • • ,tn,  Y2B(t1)B(t2) ‘ ‘ ,B(tn)U  =  r(t1)r(t2) • • *r(tn) . 

Lemma  3.4.1.  If  (B,Y2)  is  a  transient  MRP  and  (B,y2)  ~  r  then 

(1)  r  is  a  transient  renewal  process  (i.e.  r(°°)  <  1). 

(2)  The  lifetime  of  (B,y2)  has  the  same  distribution  as  the  lifetime  of 


Proof.  (1)  By  Theorem  2.2.8,  r(“>)  is  the  largest  eigenvalue  of 

B(«»)  which  is  strictly  less  than  one  since  B  is  transient. 

(2)  The  lifetime  of  a  MRP  is  the  random  variable  T^ + T2+’ * ,+Tn*« 

where  n*(oi)  *  inf  (T  . ,  (w)  =  “).  For  the  renewal  process  r,  the 
n  n+1 

distribution  of  n*  is  given  by  P(n*  **  k)  =  r(“>)k  (l-r(co)).  For  the  MRP 
(B,y2)»  the  distribution  of  n*  is  given  by  P(n**k)  =  Y2B(")k(I  -  B(°°) )u- 
Since  (B,y2)  ~  r,  Y2B(®)k(I ~  B(°°))U  =  r(«)k(l  -  r(®)) ,  so  n*  has  the  same 
distribution  for  both  processes.  Furthermore,  in  both  cases  the 
distribution  of  the  lifetime,  L(t),  is  given  by  the  convolution  of  r(t) 
with  itself  n*  times  (i.e.  L(t)  =  (1  -  r(®))  ^^r(°°)kr^  (t)  where  r^(t) 
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is  the  k-fold  convolution  of  r(t)  with  itself),  q 

One  of  the  interesting  properties  that  MRP's  with  transient 
states  can  exibit  is  equivalence  to  a  delayed  renewal  process. 

Definition  3.4.1.  Let  be  a  random  process.  If 

{S^,S2,**’}  is  a  renewal  process  and  Sq  is  independent  of  { , S2 » *  *  - } 
then  {S0,Sr...}  is  .ailed  a  delayed  renewal  process.  If  P(SQ£t)=g(t) 
and  P(Sn£t)  *  r(t),  n  =  1,2,***,  the  delayed  renewal  process  is  denoted 
(g,r). 

Definition  3.4.2.  Let  (Q,y)  be  a  MRP  on  (0,  <$T,P).  If  there  is 

a  stopping  time  n*  such  that  ^Tn*+j»^n*+2* ’ * is  a  renewal  process  with 

distribution  r(t)  and  ^Tn*+1»Tn*+2» ‘ ^ ‘ ^  is  independent  of  (T^,T2, • • • 

then  (Q,y)  ~  (g,r),  where  g(t)  -  P(T.  +  T,  +• • •+  T  *  <  t) . 

lx  n  — 

Theorem  3.4.2.  Let  (Q,y)  be  a  MRP  where  Q(t)  has  the  form 
(3.4.1).  Say  (A,Yj)  ~  r  and  A  has  core  etc.  If 

Y,B(t.)B(t7)---B(t  ,)C(t  ) 

yj  x  1 _ x _ n-i _ n 

n,tl,t2’-"’n  Y,B(t1)B(t7)---B(t  ,)C(t)U  E  ^ 

xi  x  n-i  n 

then  (Q,y)  ~  (g,r)  where  g(t)  is  the  distribution  of  the  first  exit  time 
from  the  set  of  transient  states. 

Proof.  Let  (Q,y)  be  {X^,^}  and  let  n*  =  inf{XQ  e  A}.  Clearly, 
n*  is  a  stopping  time.  Let  y^  be  the  vector 

Yj(l)  -  F(Xn.  '  i  I  Tj  +  T2  +'••+  T„.  5  t). 

If  et{  >  Vt  then  (T^  +  Tj  +•  •  •  +  T^*)  is  independent  of 

tTn*+l,Tn*+2’‘”}  and  P(Tn*+l-tl’  Tn*+2  - 12’  “  *  ,Tn*+n- V 

-  r(t, )r(t«) • * • r (t  ),  so  to  prove  that  (Q,y)  ~  (g,r)  it  suffices  to  show 
lx  n 

ej C,  Vt. 
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Let  Tx  be  the  set  of  states  in  A,  let  T2  be  the  sot  of  states  in  B, 
and  let  hn*(t)  -  P^  +  T,  +•••+  l'n*  <  t).  Thus 

•  P<Xn*  ■  l-  *0  c  fl  I  T1  +  T2  +"'+  T„*  ‘  *> 


Tctj-  +  h-;uT  I^P(n*-k)  fs-fl  Jm dt  )»<dt2) 

n  n  t^i  Jo  Jo  Jo 


+  P(Xn*  -  L-  X0  c  r2  I  Tr  +  T2  +•••  +  lnA  <  t) 
t-£t,  t-t,  t 


B(dtk_1)c(t-i;t))  it 


By  hypothysls,  this  is 

Y ,  (1) 

r +  h'Tt) 

k-1 


t-Et .  t-t,  t 

J  Z*1 


*  hVo  +  5“,V)  5}1”*"0  f  f  A2»Wtl),Wt2> 

n*  n*  T~T  Jf\ 


0  •/0 

B(dt hW'-Vi’"1,,,,..,  <*>' 

12  k-1 


where  y 


tlt2,”tk-l 


e  X.  Since  is  convex  we  have 
Y ,  (i) 


y‘W  "  iTTW  +  i.;,Tt7LP<n*'k>l,K(t)l,k*(l) 


k-1 


where  y*  e  X  So 


Y’(i) 


y,  (D  +  [h  *(t)  -  P(n*-0)]y*(i) 

i  ji" 

h_ *(tj 


where  y*  e 


X. 


Thus 


Y,  +  lh  *(t)  -  P(n*-0)  ly* 

v  i  _  __1 _ "L _ _ 

h„.(t> 


Since  YjU  -  P(n*-0)  and  y*U  -  1  we  get  y^U  -  1.  Thus  y  c  X  This 
proves  that  (Q.y)  ~  (g,r)  where  r(t)  -  y.A(t)U  and  g(t)  -  h  *(t).  n 
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Example  3.4.1.  Say  (A,y)  ~  r  and  let 


A(t) 

0 

(l-p)f (t)y 

pf  (t) 

where  f(t)  =  1-e  Xt.  Let  y *  *■  (qy,  (1-q)),  where  0  <  q  £  1. 

Claim.  (Q,y*)  ~  (g, r)  where  g(t)  =  q  +  (1-q) (1-e  ^  P^Xt). 

Proof.  Let  TQ  be  the  time  that  the  process  first  enters  the  set 
of  recurrent  states.  Since  Vn.t^.t^ • • • .t^, 

(A-q)fn~1.Cl-p)f  (t)ny  „  y 

(l-q)pn  *(l-p)f (t)nyU 

the  theorem  shows  that  (Q,y*)  ~  (g,r)  where  g(t)  =  P(Tq  -  •  But 

_oo 

P(Tq  <  t)  -  q  +  (l-q)(l-p)^pjf(j)(t)  -  q  +  (1-q) (l-e~(1'P) Xt) . 

j»l 

Thus,  with  probability  q,  (Q,y*)  is  a  renewal  process  from  time 
zero,  and  with  probability  1-q  there  is  an  exponential  delay  (while  the 
system  is  in  the  transient  state)  followed  by  a  renewal  process. 
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CHAPTER  IV 


CONVOLUTIONS  OP  MRP'S 


1.  Convolutions  of  MRP's 

Lot  fj  and  r0  bo  two  renewal  processes.  The  convolution  of 
and  r.,  is  denoted  r^*r,?  and  is  the  renewal  process  with  distribution 


r  *r.,(t) 


A 


(t  -  s)r7(ds) . 


or  equivalently 


r‘i*r2( t)  - 


r^(ds)r.,(t  -  s)  . 


The  interepoch  times  in  r  ^  *  r  ^  can  be  thought  ol‘  as  an  r ^  epoch  followed 
by  an  rn  epoch.  Clearly  rt  *r,,  »  r^*r^.  Let  r[»r2,""‘*rn  renewal 
processes.  The  renewal  process  ri*r2*‘''*rn  ^ Ined  inductively  by 


r .  *r,,*-  •  •  *r  (t)  » 
12  n 


/  rl*r2 

Jo 


?*’ ’ '*rn  _  i (d»)rH(t  -  s). 


(n) 


Definition  A. 1.1.  If  r  is  a  renewal  process  then  r  Is  the  tv 

fold  convolution  of  r  witli  inself  defined  inductively  by 

r(0)(t>  -  1 

.O) 


r  (t)  -  r(t) 

(k+1)(t)-  / 

A) 


r^k\ds)r(t  -  s) 


Let  and  m  state  MRP's  (1  <  m  <  ®) .  The  convolution  of 

Qj  and  Q.,,  denoted  can  be  defined  in  an  analogous  manner. 

% 


}j*Q2(0  ■  /  Qj(ds)t)2(t  -  s). 

Jo 


bS 


> 
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Likewise,  if  ’  *^n  are  with  the  same  state  space, 

Q 1 *Q 2 * ” ’ * *Qn  i8  fhe  MRP  with  kernel  defined  inductively 

Q1*Q2*...*gn(t)  -  ygi*Q2*---*Qn_  j(ds)Qn(t  -  s). 

Definition  4.1.2.  If  Q  is  a  MRP  then  is  the  n-fold  convolu¬ 

tion  of  Q  with  itself  defined  inductively  by 


Q(0)(t)  -  I 

q(1)(t)  -  g(t) 

.!<n+1><t)  -  % 


Q(n)<ds)Q<t  -  s). 


Again,  the  interepoch  times  in  Q^*Q2  can  be  thought  of  as  a 
epoch  followed  hy  a  Q..  epoch.  In  other  words 

lW01lj  “  ]Cy*lQ l(d8),ik[Q2(t-8)1kJ- 


Note  that  the  convolution  only  makes  sense  if  the  MRP's  have  the  same 
state  space,  and  that  Q^*Q2  a,ul  ^2*^1  are  not  Necessarily  the  same  MRP 
since  matrix  multiplication  does  not  commute. 

Notation.  The  n-fold  convolution  of  Q  with  itself  will  be  de¬ 
noted  ,  and  the  matrix  Q(t)  raised  to  the  ntl'  power  will  be  denoted 
Qn(t). 


To  prove  that  the  definition  of  convolution  of  MRP's  makes  sense 
we  need  the  following  lemma. 

Lemma  4.1.1.  Let  ^p^2,’"’,^n  MRP's  with  identical  state 

spaces.  Every  entry  of  Q, *Q.,* •••*()  (t)  is  a  nonnegative,  nondecreasing, 

1  i  n 

right  continuous  function. 

Proof.  Since  for  each  J,  is  a  MRP,  Q^(t)  is  nonnegative, 
nondecreAsing  and  right  continuous.  Thus  . .*Q^(t) 
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ni  n-1 

**•  J  =  Ql(d8l)Q2(ds2)','Vl(d8n-l)Qn(t-kf1sk)  >  0 


Assume  . .*Qk(t)  is  nondecreasing  for  k  <  n.  Then 

Ql*Q2*-..*Qn(t)  *  ^1*Q2*-..*Qn_1(t-s)Qn(ds) 


7Ql*Q2**  *  •*Qn  l(t-s)Qn(ds)  +  /qi*Q2**  •  **Qn _1(t-s)Qn(ds) 

t-A  5‘ 


where  0  <  A  <  t.  By  the  induction  hypothesis  this  is 
t-A  t 


>  /  Q1*Q2**'-*Qn_1(t-A-s)Qn(ds)  +  J Q^*.  •  •*Qn_1(t-s)Qn(ds) 

Q  t-A 

t-A 

—  /  Qx*Q2*' * ■*Qn_i(t-A_s^Qn^ds)  =  Q,*Q2** • -*Qn(t-A). 

(T 

Assume  is  right  continuous  for  k  <  n.  Let 

{t  }  j-1,2,***  be  a  decreasing  sequence  with  lim  t.*t.  To  show 

J  j-x»  J 

^1*^2*"  *  "*^n^  continuous  it  is  sufficient  to  show  that 


lim  Q1*Q2*. .  •*Qn(t.)  *  Q].*Q2*' *  ,*QnCt). 
j-**  2 


f) 

lim  Q-*Q0*...*Q  (t.)  *  lim  I  .. (t .~s)Q  (ds) 

j-*»  1  2  n  J  J-ko  JQ  1  2  n_1  J  n 

lim  /q.*Q,*.  ••*(},  (t.-s)Q  (ds)  +  lim  Q  *Q,*- •  **Q  (t  -s)Q  (ds) 

J-  <f  1  2  k"1  j  n  j—  {  1  2  n_1  3  n 


By  the  Monotone  Convergence  Theorem,  this  is 


/lim  Qj*Q2*’ ' **Qn_1(tj-s)Qn(ds) + J'  Qj*Q2** - •*Qn_1Ct j-s)Qn(ds) 
j-»®  j-+®  t 


Since  Qj*Q2*‘ * **Q  ^(t)  is  right  continuous. 
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^lim  Q1*Q2*...*Qn_1(tj-s)Qn(da)  -  *  '*Qn-l(t"s)Qn(ds) 

“  Q1*Q2*' * -*On(8). 

Since  Q^*Q2** • **Qn_^(t)  is  nonnegative  nondecreasing  and  bound¬ 


ed. 


r i 

0  <  lim  J  Q1*Q2*' * '*Qn _1(tj-s)Qn(ds) 

<  lim  /  Q 
j-~  t 


*Q2**  ••*Qn_j(°“)Qn(ds) 


-  Q1*Q2*---*Qn_1(“)  lim  (Qn(t. )  -  Q  (t))  -  0 
Thus  lim  <},*<},*••  **Q  (t.)  -  Q,*Q9*-*-*Q  (t).  □ 

j-x»  A  n  J  1  z  n 

Lemma  A.  1.2.  Qi*^2*’  *  •*Qn(C0)  “  Q1(“)Q2(")  •  •  *Qn(“)  • 

Proof.  Assume  Qj**^*'  *  ■  Q1(“)Q2(°°)  *  • 'Q^C”)  for  k  <  n. 

Since  Qj*Q2*’ • **Qn(t)  is  right  continuous  and  bounded. 


t-*» 


h 


Ql*Q2*’ *  **Qn(”)  "  lim  /Q i*Q2**  * •*Qn_1(t-s)Qn(ds) 


/»  oo 

q1*Q„**«-*Q  1 (t-s)Q  (ds)  -  lim  /qi*Q9*--**Q  . (t-s)Q  (ds) 

12  n_1  n  t-*-  {  1  2  n_1 

By  the  Monotone  Convergence  Theorem  and  the  induction  hypothesis, 

r  T 

If®  /  Qj*Q2*‘ • •*Qn_1(t-s)Qn(ds)  -  /  lim  Q1*Q2*- • •*Qn_1(t-s)Qn(ds) 

t-KD  0  0  t-**> 

00 

•  Q1(“)Q2(“’)* 

0 

-  Q1(”)Q2(®)'-*Qn(“). 
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Since  Qj*Q2** • **Qn_^(t)  is  nondecreasing  and  bounded 

oo 

°  <  lim  y'Q1*Q2*-..*Qn_1(t-s)Qn(ds) 

oo 

<  lim  ^Q1*Q2**  •  •*Qn_1(“)Qn(ds) 


“  Q1(“)Q2(«)**,Qn_1(<»)  lim  (Q(<*>)  -  Q(t))  -  0. 

t-X“ 

Thus  •  •*Qn(®)  «  Q1(“)Q2(»)  •  • -Qn(“) .  o 

Coirollarj'  4. 1.2.1.  If  ‘  *  »Qn  are  Persistent  then 

Ql*Q2*---*Qn(»)U  *  U. 

Proof .  Since  (»)U  «  U  for  each  j, 

VQ2**‘-*Qn(")U  *  Qn(«>w  “  u-  D 

Lemmas  4.1.1,  4.1.2  and  corollary  4. 1.2.1  show  that 
Ql*Q2*..-*Qn(t)  is  the  kernel  of  a  MRP.  We  also  have 

Corollary  4. 1.2. 2.  If  Q  has  steady  state  vector  n  then  also 

has  steady  state  vector  it. 

Proof .  TrQ^(“>)  **  irQ11  («*>)  ■  it.  o 

In  general,  the  canonical  form  of  Q^*Q2  will  not  be  the  same  as 
Ql  °t  Q2*  In  fact  knowing  just  the  canonical  form  of  and  Q2  is  not 
enough  to  determine  the  canonical  form  of  Qj*Q2.  For  example,  say 


(where  X  is  a  nonzero  function  of  t).  Qj  and  Q2  are  both  irreducible 
and  aperiodic  but  Q^*Q2  is  reducible. 


4 


Q^U)  - 


Lemma  4.1.3.  If  Q  is  an  irreducible,  aperiodic,  non-null  MRP 
then  so  is  Q^. 

Proof.  Since  Q  is  irreducible  and  non-null  the  steady  state 

vector  ti  has  all  entries  positive.  Also,  lim  Q  (<»)  -  Un.  Thus,  Ve  >  0 

k-*°° 

and  Vi,j  there  is  a  k^Ce)  such  that 

Vk  >  k1J(e),  lQ1j't(“)  ~  Hjl  <  e. 

Choose  c  <  ir^  and  ^ar8e  enough  so  that  k'n  >  k^(e).  Thus, 

,Jt'n,  .  „  _  .  „k'(n+l),  .  .  _ 

Qii  (“)  >  0  and  (")  >  0. 

Thus,  the  greatest  common  divisor  of  {k:  [Q^n^(°»)]^  >  0}  is  one.  Since 
this  can  be  done  for  every  state,  is  aperiodic. 

Choose  e  <  Ttj  and  k'  large  enough  so  that  k'n  >  k^j(e).  Thus 
(Q^  (“)  1  >  0,  so  it  is  possible  to  get  from  state  i  to  state  j  in 

Q^n\  Since  this  can  be  done  for  any  i  and  j,  is  irreducible,  il 


2.  Convolutions  and  Equivalence. 

Ideally  we  would  like  ~  r^.Q,,  ~  r2'*“,^n~  rn  to  imP^y  that 
^1*^2*" * "*^n  ^  rj*r2*' • •*rn.  Unfortunately,  the  situation  is  not  quite 
that  simple.  Consider  the  following  example. 


s 
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By  theorem  2.2.2,  ~  where  r^(t) 


0  if  t  <  1, 

■|  if  1  <  t  <  2, 
1  if  t  >  2, 


and  by  example  2.2.1,  Q2  ~  r2  where  r2(t)  * 


0  if  t  <  1, 

\  if  1  <  t  <  2, 
1  if  t  >  2. 


Direct  calculations  yield  r^*r2(3)  -  and 

1  1  J_ 

6  3  18 

Q1*Q2(3)  =  [  g  6  18 

1  1  _5_ 

■  6  3  18 


It  turns  out  that  ^  is  not  an  eigenvalue  of  Qj*Q2(3)  so  by  theorem 

2.2.8,  Q1*Q2  r1*r2* 

Theorem  A. 2.1.  If  Q  ~  r  then 

Proof.  Let  CfC  satisfy  J^Q(t)  C  r(t )£f{  ,  Vt.  Say 
Jft(k)(t)C  r(k)(t)J^,  Vt,  Vk  <  n.  Then 


J^Q(n)(t)  -  Jfj Q(n-1)(t-s)Q(ds) 

C  ^'r(n"1)(t-s)r(ds)J^ 


-  r  (n)  . 

Thus  Jfa(n)(t)  C  r(n)(t)J^,  so  Q(n)~  r(n).a 

Corollary  A. 2. 1.1.  If  (Q»y)  ~  r  then  (Q^  \y)  ~  r^ 

Proof .  Say  (Q^k\y)  ~  for  k  <  n,  and  yQ^(t)  e  r^(t)J^  , 


Vt.  Then, 


73 


,(n) , 


Y/‘ 


Y<T  '(t)  *  Y/<r  '  (t-s)Q(ds) 


(n-1) , 


e  /  r^n_1^(t-s)r(ds)J^ 


/' 


-  r<n)(t)Jir. 

Thus  (Q<n,,T)  ~  t(n)  and  fQ<n><t)  t  r<n)(t)J£\  □ 

Corollary  4. 2. 1.2.  Let  Q^,  j  *  1,2, • • * ,n  be  MRP's  and  assume 
there  is  a  set  £/{  that  satisfies  t^Qj  (t)  C  r^(t)Jjf  ,  Vt,  V j .  Then 

Ql*°2*...*Qn~  ri*r2*---*rn. 

Proof.  By  hypothesis,  ~  r^.  Say  ~  ri*r2*”‘*rk 

and  J^Qj*Q2*.  .  .*Q^(t)  C  r^*r2**  •  ,  Vt,  Vk  <  n.  Then 


JifQ1*Q2*---*Qn(t)  -  •  •*Qn_1(t-s)Qn(ds) 

C  ^i*r2*"'*rn-l(t"3)rn(ds)'>1^ 


=  r2*r2*- • »*r  (t)Jjf  . 


Thus,  Q^*Q2* • • • *QR  ~  ri*r2*" * '*rn'  a 

Corollary  4. 2. 1.3.  Let  Q  and  Y  be  MRP's.  If  Q  ~  Y  then 
Q(n)  F  Y(n) 

F 

Proof .  By  theorem  2.6.4,  Q  ~  Y  implies  there  is  a  set  of 
matrices,  oU,  of  the  form 
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(where  each  row  is  nonnegative  and  sums  to  one)  so  that 
«y^Q(t)  C  'l(X\Al »  Vt.  Say  for  k  <  n,  and 

./^Q(k)(t)C  Y(k)(t)^,  Vt.  Then 

^Q(n)(t)  »  t^/Q(n'1)(t-s)Q(ds) 
t» 

C  /Y(n)(t-s)Y(ds)t^'  =  Y(n)(t)^, 


(n)  F  (n) 
so  Q  ~  Y 


D 


The  MRP  ***^n  Can  ana^yze<^  a  different  manner. 

Consider  the  MRP  with  kernel 


Let  {A^,A2, *  *  * ,An>  be  a  partition  of  the  states  of  Q  where  Aj  is  the 
collection  of  states  corresponding  to  Qj.  Let  f  be  the  map  that  lumps 
the  states  of  Q  to  {A^.A^, • • • >An>  and  let 


Proof .  From  Corollary  4. 2. 1.3  we  know  that  ~ 


The 


form  of  Q  and  Y  yields 


Ql*Q2*- • **Qn(t) 

* 

VV-'-'Vi't) 

and 


r, fcr,,** • • *r  (t) 
1  /  n 

r2*r3*’ * 

• 

• 

r  *r,*. .  .*r  ,(t) 

n  1  n-1 

The  theorem  follows  by  observing  the  top  left  components  of  Q 


(n) 


and 
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with  theorem  3.2.2  in  mind,  o 

Corollary  4. 2. 2.1.  If  Q  ~  Y  then  Vj, 
•  *r  . 


Wi*‘ 


*Qj-l ~  rl*r2*"  „ 

Proof.  Since  rj*rj+i*' • »*rj  ^(t) 


rl*r2*‘ 


•*rn(t),  the  remain¬ 


ing  components  of  and  Y^  give  the  result.  Q 


Clearly,  any  sufficient  condition  for  Q  ~  Y  is  sufficient  for 
Qi*Q2*' • • *Qn ~  rl*r2*" * **rn' 

Theorem  4.2.3.  If  there  exists  Y^»Y2>'*'*Yn  such  that  Vj, 
YjQj(t)  -  rj(t)Y;j+1.  Vt,  then  Q1*Q2*«  • -*Qn  ~  r1*r2**  * -*rn. 

Proof .  Let 


Y1 

Y2 

• 

• 

» 

Yn 

By  hypothesis. 


rQ(t)  - 


so 


F 

Q  ~ 


Y  by  theorem  2.7.2.  o 


Y(t)T, 


3.  Markov-Renewal  Functions  and  Markov-Renewal  Equations. 

00 

Definition  4.3.1.  Say  Q  is  a  MRP.  Let  RQ(t)  «  ]^Q(n)<t). 

W  „  n-0 

Similarly,  if  r  is  a  renewal  process  then  Rr(t)  -  ^r^n\t).  Rp(t)  is 

n“0 

called  the  Markov-renewal  function  associated  with  Q  and  Rr(t)  is  called 
the  renewal  function  associated  with  r. 


In  this  section  we  will  only  deal  with  irreducible,  persistant 


MRP's.  It  can  be  shown  [3]  that  Rg(t)  is  well  defined  and  finite.  Let 
(Q,y)  be  defined  on  P),  and  let 


H(q  -  sup  tn 


:  £v“ 


>  <  ti 


V,(t)  ■  E("(Q,Y)<t--))- 

Similarly,  if  r  is  defined  on  (Q,  «9?",P)  let  Nf(t)  be  defined  analogously. 

For  a  MRP  (or  a  renewal  process)  let  us  call  the  set  of  points 
n 

{N  T,  (u>) :  n**l,2,’”},  events.  Thus  N/r.  .(t)  and  Nr(t)  are  the 

P J  w>1,) 

expected  number  of  events  in  the  interval  [0,t]  for  the  corresponding 
random  process  (we  assume  there  is  an  event  at  time  zero).  It  is  well 
known  [3]  that  Nf(t)  -  Rf(t)  and  ^  (t)  -  YRg(t)l).  Thus,  theorem 

4.2.1  has  an  important  corollary. 

Theorem  4.3.1.  If  (Q,y)  ~  r  the’  ^  (t)  ■  Nr(t). 

Proof .  Corollary  4. 2. 1.1  states  that  (Q^,y)  ~  r  for  every  n. 


so 


yQ 


<n)(t)U  -  rln,(t). 


Thus, 


00  oo 

n(q.y>  (t) "  YRQ(t)u  “  r(y;Q(n)(C))u  -  y\ 

n-0  n-0 


(n) 


(t)  -  Rr(t) .  o 


Actually,  theorem  4,3.1  can  be  strengthened  slightly.  If  Q  has 
core  then  ^(t)  -  Nf(t)  for  every  B  e  ${  . 

Definition  4.3.2.  Let  Q  be  an  n  state  MRP  (1  <  n  <  «),  and  let 
gXt)  be  a  column  vector  of  n  nonnegative  functions  bounded  on  finite 
intervals.  An  equation  of  the  form 
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(4.3.1) 


/n  ^  r  /\ 

f(t)  -  g(t)  +  J Q(ds)f(t-s) 
0 


/N 


(where  f(t)  ia  an  unknown  column  vector  of  functions)  is  called  a 
Markov  renewal  equation. 

If  r  is  a  renewal  process  and  g(t)  is  a  scalar  function  bounded 
on  finite  intervals  then 


(4.3.2) 


f(t)  -  g(t)  +  Jr(da)i  (t-s) 


(where  f(t)  is  an  unknown  scalar  function)  is  called  a  renewal  equation. 
The  solution  of  (4.3.2)  is  unique  and  is  given  by 

(4.3.3)  f(t)  -  /R_(ds)g(t-s) , 


•  A1 


Lemma  4.3.2.  The  solution  of  (4.3.1)  is  unique  and  is  given  by 

(4.3.4)  f(t)  -  yj^dsygU-s). 

Proof.  Since  Q  is  irreducible  and  persistent  the  lemma  follows 

from  [3,  Chap.  10],  a 
til 

The  j  element  of  f(t)  can  be  thought  of  as  the  value  of  some¬ 
thing  at  time  t  given  that  Q  is  in  state  j  at  time  zero.  Thus,  if  we 
start  with  initial  distribution  y,  the  expected  value  is 

(4.3.5)  YfU)  -  ^(ds^t-s). 


Let  x  be  the  steady  state  vector  for  Q  and  assume  Q  ~  r.  It 
would  be  nice  if  it  were  always  true  that 


■  A' 


Wf(t)  “  / R_(ds)  (TTg(t-s)) 

since  this  involves  only  scalar  integration.  Unfortunately  the  situation 


is  not  quite  that  simple. 


mmmmmmrnm 
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Theorem  4.3.3.  nf(t) 


(ds)  (irg^t-s) )  for  all  bounded 


nonnegative  g  if  and  only  if  TtRq(t)  «*  Rr(t)u,  V  t. 


Proof.  ( ) .  irf 


(t)  “  JvRq(.ds)/g(t-s)  -  yRr(d8)(irg'(t-s)). 

0  0 

■»■)  Say  there  exists  a  t  such  that  irRq(t)  j*  R^(t)it.  Then  for 
some  j,  [TrRQ(t)]j  +  R^t)^.  Since  RQ(t)  and  Rr(t)  are  right  continuous, 

there  exists  e  >  0  such  that 

t+e  t+e 

J  [irR^Cds)  ]  j  i  J  Rr(ds)Tr^. 


Define 'g'’  as  follows: 


n  * 


Thus, 


g’(t>  - 


0  if  i  +  j, 

1  if  i  *  j,  0  <  t  <  E, 
0  if  i  -  j,  t  >  e. 

t+E 


t+e  t+e 

irf(t  +  e)  -  J~  Rr(ds)  (irg(t  +  e  -  s))  -  /  Rr  (ds)iTj 


t+e  t+e 

i  f  [irRQ(ds)]j  -  J  TrRQ(ds^’(t  +  e  -  s) 

y  rv 


/\ 

*  itf (t  +  e) , 


which  is  a  contradiction. a 

Corollary  4. 3. 3.1.  If  irQ(t)  -  r(t)n,  Vt,  then 


irf^Ct)  -  ^Rr 


(ds)  (iTgXt  -  s)) , 


Proof.  First  we  show  that  7rQ^(t)  “  rvn;(t)fl,  V  t,  Vn.  Assume 


(n) , 


it  is  true  for  k  <  n.  Then 
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irQ^(t)  ■  n  /Q<n  ^(t-s)Q(ds)  ■  fx^n  ^  (t-s)r (ds)it  ■  r^(t)ir. 


■/* 


(n-1) , 


»»  00 

Thus,  itRg(t)  ■  n  ^  (t)  "  ^  (t)n  ■  Rr(t)ir,  so  the  result  holds. □ 


Thus,  if  nQ(t)  »  r(t)ir,  Vt,  the  solution  of  any  Markov- renewal 
equation  in  steady  state  is  the  same  as  the  solution  of  the  correspond¬ 
ing  renewal  equation.  Unfortunately,  irQ(t)  -  r(t)n,  Vt,  is  not  a 
necessary  condition  for  Q  ~  r. 

Let  Z  ^n^(t)  *  Q^(x+t),  and  let  Z  be  the  ring  of  matrix 
valued  functions  generated  by  {Z  ^(*)  :  x  e  K,  n*  0,1,2,  •••}  under  the 
usual  operations  of  matrix  multiplication  and  addition.  If  Q  has  core 
then  for  any  y  e  Cf{  and  Z(*)  e  Z  (closure  of  Z  in  the  usual  sense) 
we  have  yZ(t)U  ■  irZ(t)U,  where  n  is  the  steady  state  vector  for  Q. 

Theorem  4. 3. A.  If  Q  ~  r  and  ^(t)  »  Z(t)U  where  Z(-)  e  Z  then 


V^r  e  CK,  yHt) 


.(ds)(YgU-s)). 


Thus, 


Proof.  Since 'gtt)  -  Z(t)U  we  have  yg^t)  -  Y^gtt),  V  Y,  Y*  t  £f{ . 


yf'it)  -  ^YRn(ds)'g'(t-s) 


IS 


yQ^  (dsXgU-s). 


Since  YQ^n^(ds)  ■  r^(ds)Y  ,  where  y  c  CK  we  have 


yi(t) 


-  /^r^n^(ds)(Y^gXt-s>) 

0  n*0 


.(ds) (Y8tt-s)) .  o 
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Although  it  appears  that  theorem  4. 3. A  imposes  a  large  restric¬ 
tion  on  the  number  of  Markov- renewal  equations  we  can  consider,  it  should 
become  clear  from  the  following  examples  that  in  many  problems  of 
interest,  ‘gXt)  will  have  that  form.  In  the  following  examples,  Q  ~  r 
and  Q  has  steady  state  vector  tt. 

Example  4.3.1.  Let 


Px(t)^  “  Pr  [no  events  in  the  interval  (t,t  +  x]|Q  starts  in  state  i], 
Then 


/\ 


P„(t)  *=  (I  -  Q(t  +  x))U  +  /Q(ds)P  (t  -  s) 


/« 


/\ 


Since  (I  -  Q(t  +  x))  e  Z,  we  know  that 

t. 


-  ^Rr(ds)u(l- 


Q(t  +  x  -  s)  )U 


(ds)  (1  -  r(t  +  x  -  s) ) , 


Example  4.3.2.  Let 


^PVt).  *  Pr  [exactly  one  event  in  (t,  t  +  x]|Q  starts  in  state  i] . 

X  1 

Then, 

*** 

J (ds ) (I  -  Q(t  +  x  -  s))U  + 


-  f  q(ds)  (I  -  Q(t  +  x  -  s))U  +  /Q(ds)^<t-s). 

r 


t+X 

Since  J  Q(ds)  (I  -  Q(t  -  x  -  s))  e  Z  ,  we  have 


i 
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t_  t+X-8 

i^(t)  -  ^R_(ds)n 

t  t+x- 

■  A(d9)  i 


Q(du)  (1  -  Q(t  +  x  -  u)) 


t+x-s 


r(du) (1  -  r (t  +  x  -  u) ) . 


Example  4.3.3.  let  N(t,u>)  be  the  number  of  events  up  to  time  t 
(including  the  events  at  0),  and  let  {T^OdJ.Tj (<»>),•  *  • }  be  the  interepoch 
times.  The  forward  recurrence  time  at  t,  F^(u>)  is  defined  to  be  the  time 

N(t,u>) 

until  the  next  event  starting  at  t  (i.e.  Ft(u>)  «*  (  T  (to))-t).  let 

J-l 

/\  . 

fx(t)1  ■  Pr(Ft  ^  x|Q  starts  in  state  i) . 


^x(t)  -  (Q(t  +  x)  -  Q(t))U  +  -  s) , 


c(t)  *  ^r(ds)(r(t  +  x  -  s)  -  r(t  -  s)). 


N(t,w)-1 

Example  4.3.4.  let  Bfc (u>)  »  t  -  T.  (os),  be  the  backwards 

j-l 


recurrence  time,  let 


^(t)  *  Pr(Bt  >  x|Q  starts  in  state  i). 


t 

\(t)  -  (I-Q(t))I{c  >x}  +  /QCds^Ct-s). 


Since  (X  -  QCt))l{t  >  x}  -  <I-Q<t))Q(0)(t-x)  e  Z, 


CHAPTER  V 


DISCUSSION 


1.  Summary 

The  study  of  equivalent  MRP's  has  its  roots  in  the  study  of 
functions  of  Markov  chains.  Burke  and  Rosenblatt  [1]  gave  conditions 
for  functions  of  a  Markov  chain  to  be  Markov.  This  came  to  be  know 
as  strong  and  weak  lumpabillty.  Kemeny  and  Snell  [8]  gives  the  best 
discussion  of  this  topic.  It  can  also  be  found  in  Rosenblatt  (14]. 
Serfozo  [15],  (16]  showed  that  the  concept  of  lumpabillty  in  Markov 
chains  extends  easily  to  MRP's.  These  concepts  are  apparently  considered 
unimportant  by  the  masses  since  there  has  been  very  little  reference  to 
them  in  the  literature  since  1972. 

The  reason  for  the  lack  of  interest  is  probably  that  strong 
lumpabillty  is  too  strong  a  condition  to  be  useful,  and  nobody  has  ever 
considered  in  detail  the  idea  of  weak  lumpabillty  to  a  renewal  process. 
What  has  been  shown  here  is  that  these  concepts,  in  an  appropriate 
modified  form,  are  important  both  in  applications  and  in  the  foundational 
study  of  MRP's.  It  would  seem  that  any  thorough  discussion  of  MRP's 
would  contain  a  section  discussing  the  idea  of  the  "time"  processes  of 
two  MRP's  having  the  same  distribution.  This  has  not  been  the  case.  The 
first  discussion  of  this  concept  is  found  here  in  Chapter  II  and  III. 

There  is  a  better  reason  for  studying  equivalence  than  the 
intuitive  feeling  that  it  should  be  done.  Equivalence  is  exactly  the 
condition  for  being  able  to  substitute  one  MRP  for  another  in  (say)  a 
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queueing  network  without  changing  the  character  of  the  system. 

Examples  of  the  use  of  this  sort  of  tactic  for  simplifying  the 
analysis  of  queueing  systems  abound,  although  until  now  there  has  been 
no  theory  that  encompasses  all  of  them.  The  best  known  example  is  that 
the  output  from  a  steady  state  M/M/1  queue  is  a  Poisson  process  (as 
opposed  to  the  infinite  state  MRP  by  which  it  is  most  naturally 
characterized).  More  results  of  this  form  can  be  found  in  Disney  et  al. 
[4]  and  Laslett  [9].  There  are  also  results  that  show  certain  flow 
processes  in  Jackson  networks  are  Poisson  [12].  The  value  of  these  kinds 
of  results  should  be  clear.  They  make  it  possible  to  analyze  systems 
that  would  otherwise  be  intractable.  As  a  simple  example,  consider  two 
queues  in  tandem  where  the  first  queue  is  M/D/1/1  and  the  second  is 
•/M/l/«>.  Since  the  output  from  the  M/D/1/1  queue  is  a  renewal  process  in 
steady  state,  the  second  queue  is  a  G/M/l/»  queue,  which  is  well  under¬ 
stood.  We  can  easily  write  down  the  steady  state  queue  length  distribu¬ 
tions,  waiting  times,  and  other  quantities  of  interest.  This  would  be 
impossible  if  it  were  not  known  that  the  output  from  the  M/D/1/1  queue 
is  renewal. 

From  the  results  of  Chapter  II,  one  can  quickly  verify  all  of 
these  known  results.  One  can  also  settle  some  previously  unsolved 
problems  (example  2.4.4)  and  come  up  with  curious  and  unexpected  results 
such  as  in  examples  2.4.5  and  2.4.6. 

Section  2.4  has  shown  that  the  tools  developed  in  Chapter  II 
are  useful  for  solving  important  problems,  especially  problems  dealing 
with  flows  in  queueing  networks.  The  emphasis  in  tills  paper,  though, 
was  on  the  development  of  the  theory  of  equivalent  MRP's.  More 
specifically,  the  emphasis  was  on  equivalence  between  a  finite  or  count- 
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able  state  MRP  and  a  renewal  process. 

In  Chapter  II,  necessary,  sufficient  and  necessary  and  sufficient 
conditions  were  given  for  (Q,y)  ~  r.  The  ideas  of  collapsibi 1 ity  and 
equivalence  between  two  finite  or  countable  state  MRP's  were  discussed 
and  conditions  for  them  were  given. 

In  Chapter  III  the  various  canonical  forms  of  MRF"s  were 
examined  in  detail.  Reducible  and  periodic  MRP's  were  analyzed  in  terms 
of  their  components.  One  interesting  observation  was  that  a  weighted 
average  of  steady  state  MRP's  can  only  be  renewal  if  each  MRP  is 
equivalent  to  the  same  renewal  process  (Theorem  3.2.3).  It  is  likely 
(though  unproven)  that  the  same  conclusion  holds  even  if  the  MRP's  are 
not  in  steady  state  (conjectures  3.2.1,  3.2.2).  It  would  also  be  very 
interesting  if  these  conjectures  turned  out  to  be  false. 

For  the  general  finite  or  countable  state  MRP,  answers  were 
given  to  the  questions 

(1)  For  a  given  Q,  which  r  can  satisfy  Q  ~  r2 

(2)  For  a  given  Q  and  r  which  y  can  satisfy  (Q,y)  ~  r? 

Also,  equivalence  between  a  MRP  and  a  delayed  renewal  process 
was  defined  and  conditions  were  established  for  it. 

Chapter  IV  was  a  look  at  the  idea  of  convolutions  of  MRP's.  The 
goal  was  to  determine  when  equivalence  is  preserved  under  convolution. 


It  became  obvious  that  the  answer  is  not  simple.  tt  is  easy  to  produce 
an  example  of  ~  r^,  Qj  ~  r^.  *'  rj*r2*  although  conditions  can 


be  given  to  assure  ~  r^*r^.  One  computational  trick  is  to 

realise  that  ”*^n  can  as  a  periodic  MRP  whore 


the  Qj'*  are  stages.  The  solution  of 
the  convolution  of  a  MRP  with  Itself. 


Markov  renewal  equations  involves 
One  would  hope  that  if  Q  ~  r,  the 
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solution  of  a  Markov  renewal  equation  involving  Q  could  be  simplified. 
This  idea  is  made  precise,  and  although  not  all  Markov  renewal  equations 
can  be  simplified,  it  is  shown  that  a  large  and  important  class  can  be. 

2.  Extensions. 

Consider  a  renewal  stream  of  customers  entering  a  queueing  system 
of  n  exponential  servers  in  tandem  (each  with  infinite  waiting  room). 
There  is  no  problem  analyzing  the  first  queue.  It  is  a  G/M/l  queue,  and 
its  properties  are  well  known.  Unfortunately,  the  behavior  at  the  second 
queue  is  almost  impossible  to  analyze  since  the  output  from  a  C/M/1  queue 
is  not  a  renewal  process.  In  fact,  according  to  the  definitions  in  this 
paper,  it  is  not  even  a  MRP.  In  order  to  get  the  distribution  of  the 
time  between  one  departure  and  the  next,  one  must  know  whether  the  queue 
is  empty  or  not.  If  it  is  empty,  one  still  needs  to  know  the  distribu¬ 
tion  of  the  time  until  the  next  arrival  in  order  to  get  the  distribution 
of  the  time  until  the  next  departure.  The  foward  recurrence  time,  except 
for  the  Poisson  process,  is  a  nontrivial  function  of  a  continuous  param¬ 
eter,  t  c  (0,<»).  Thus,  the  only  Markov  chain  one  can  find  embedded  at 
departure  points  must  keep  track  of  the  queue  length  and  the  time  since 
the  last  arrival  (i.e.  the  state  space  is  [0,®]  x  {0, 1, 2, • • • )) .  The  out¬ 
put  from  the  second  queue  is  even  worse.  Again,  the  state  space  must 
include  the  queue  length  and  the  time  since  the  last  arrival.  Unfortu¬ 
nately,  the  foward  recurrence  time  for  the  input  to  the  second  queue  is 
the  foward  recurrence  time  of  a  MRP  with  state  space  10,®] x {0,1,2,* •• }. 
Thus,  the  departure  process  from  the  second  queue  is  described  by  a  MRP 
with  state  space  [0,®]  x  (0,1,2, •*•} x {0, 1,2, *** }.  (i.e.  must  keep  track 
of  the  queue  length  at  the  first  and  second  queue,  and  how  long  it  has 
been  since  the  last  arrival  to  the  first  queue).  The  state  space  for  the 
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departure  process  from  the  nC^  queue  is  [0,»]  x  {0, 1,2, • • • }n. 

Although  finding  the  steady  state  queue  length  distribution  (or 

even  approximating  it)  in  this  very  simple  queueing  network  is  a  very 

difficult  problem,  the  fact  is  that  if  any  method  is  developed  to  deal 

with  this  problem  the  same  method  could  probably  be  used  to  solve  for 

queue  length  distributions  in  an  arbitrary  network  with  independent 

renewal  arrivals  and  servers.  This  is  because  in  a  network  of  that  form 

the  state  space  needed  to  describe  the  flow  of  traffic  on  any  arc  in 

nl  n2 

the  network  can  never  be  worse  than  [0,  «° ]  x  (0,1,2, •••} 

The  concept  of  equivalence  between  a  MRP  and  a  renewal  process 

can  easily  be  extended  to  MRP's  with  general  state  space,  and  state 
nl  ,n2 

spaces  like  [0,®]  x  {0,1,2, •••}  “  pose  no  problem  at  all.  Let 

{S.&)  be  a  measure  space  and  let  (fi,  &  ,P)  be  a  probability  space. 

Let  X  :  0  -*■  S  be  ^-measurable  and  let  T  :  ft  -+  (0,“]  be 
n  n 

-measurable .  Then  {X  ,  T  }  is  a  MRP  with  state  space  (S,,^)  if 

n  n 

for  each  t  there  is  a  Q^Ct):  S  x  (0,1]  such  that 


P(X  e  A,T  < 
n  n  - 


t  X 


-l’Xn-2’ 


,X0’^n-l’^n-2’ 


•V  =  St  A(t) 
n-1 


a.  s . 


where  Q  (t)  is  measurable  for  fixed  A,  and  Q  t (t)  is  a  measure  on 

nl  n2 

for  fixed  s.  In  queueing  networks  S  r  [0,®]  x  {0,1,2,***}  and 

m  Xt/f  where  t/f'  m  2^’^’^’  The  initial  distribution 

of  the  MRP  is  a  measure  p  on  (S,  where  p(A)  ■  P(Xq  e  A) .  We  can 
denote  a  MRP  on  (S,  )  with  kernel  Q(t)  and  initial  distribution  p  by 

(Q,p).  Let  r  be  a  renewal  process.  We  say  (Q,p)  ~  r  if 


Vn,t1,t2,-*-,tn,  P(T1  <  tlfT2  <  t2,---,Tn  <  tn)  ■  r(t1)r(t2)---r(tn) 


which  implies 


- _ L 
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V  n,  ^^2* 


1,(dV<)s0ds1(tl>\ds2<t2>' 


3s  ds 
n-1  n 


=  r(t1)r(t2)‘ • *r(tn). 

It  seems  clear  that  the  conditions  and  properties  for  equivalence 
in  this  more  general  context  should  be  analogous  to  those  given  in  this 
paper. 


3 .  Generalizations. 

There  is  at  least  one  direction  that  the  concept  of  equivalence 
between  MRP's  can  be  generalized  that  could  prove  to  be  invaluable  in  the 
applied  study  of  random  processes.  Equivalence  is  the  condition  that  en¬ 
ables  one  to  substitute  one  MRP  for  another  without  changing  the  char¬ 
acter  of  the  system  at  all.  If  there  were  some  metric  on  the  space  of 
MRP's,  one  would  expect  equivalent  MRP's  to  be  zero  distance  apart  (i.e. 
the  same  point)  in  that  space.  The  general  notion  referred  to  above  is 
this  metric  on  equivalence  classes  of  MRP's.  Ideally,  this  metric  would 
be  easy  to  compute  and  a  large  class  of  queues  (thought  of  as  operators 
on  the  space  of  MRP's)  would  be  continuous  operators. 

First  of  all,  it  is  possible  to  define  a  metric  on  the  space  of 
MRP's;  for  instance  the  Levy  metric  [10] 


d((Q1,Y1),(Q2,Y2))  “  inftF^B)  < :F2(B  )  +  e,F2(B)  <  Fj  0^)  +  e,  Vb  e 

e>0 

where are  the  Borel  sets  in  R  ,  B^  is  the  open  e-neighborhood  around 

00 

B,  and  F^  is  the  measure  on  defined  by 


Fj<Tl  ^  Vt2 


<  t 


-vjqj(ti)Qj(t2)...qJ<t0Hl. 


Since  the  Levy  metric  induces  the  topology  of  weak  convergence, 
results  such  as  Whitt  [18]  show  that  a  large  class  of  queues  are 
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continuous  operators  in  this  metric.  The  problem  is  computing  or 
approximately  computing  distances  in  the  metric.  If  this  can  be  done 
it  might  be  possible  to  get  bounds  on  the  error  when  one  MRP  is  sub¬ 
stituted  for  another.  One  suspects  that  if  two  queues  are  identical 
except  for  the  arrival  streams,  behavior  of  the  queues  will  be  "close” 
if  the  two  arrival  streams  are  "close"  in  the  metric.  For  example 
consider  two  queues  in  tandem  where  the  first  is  M/M/l/N  and  the  second 
is  */M/l.  If  N  <  00  the  arrival  process  to  the  second  queue  is  not  a 
renewal  process,  and  the  steady  state  queue  length  at  the  second  queue 
will  not  be  geometric  as  it  would  be  if  N  =  “>.  For  large  N,  though,  one 
would  expect  to  find  little  difference  between  the  true  steady  state 
distribution  and  the  geometric.  (see  example  2.4.3) 

4.  Abstractions 

Some  of  the  rich  mathematical  structure  of  the  space  of  MRP's 
should  be  pointed  out,  although  no  attempt  will  be  made  here  to  exploit 
these  ideas. 

First  of  all,  corresponding  to  each  N  <  »  is  the  space  of  N 
state  MRP’s.  This  space  can  be  furthe  '  broken  down  to  the  space  of  N 
state  MRP’s  with  initial  distribution  y,  y  e  The  space  of  N  state 

MRP's  with  initial  distribution  y  is  a  semigroup  under  at  least  two 
different  forms  of  multiplication.  The  results  of  Chapter  IV  show  that 
the  space  is  a  semigroup  under  convolution,  and  it  is  easy  to  verify 
that  (Q^y)  •  (Q2»  Y)  “  (Q3.Y)  where  Q3(t)  =  Q1(t)Q,(t)  is  a  multiplication. 
Both  of  these  semigroups  are  noncommutitive. 

The  space  of  finite  or  countable  state  MRP's  is  also  a  metric 
space  where  d((Q1,y1),  (Q2y2))  =  0  if  (Q1,y1>  ~  (Q2.Y2>.  By  allowing 
more  general  state  spaces,  the  space  might  be  complete  with  respect  to 
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the  metric,  A  reasonable  conjecture  is  that  the  space  of  MRP's  with  state 
spaces  in  is  complete  under  the  Levy  metric.  The  topology  of  this 
space  (or  even  the  space  of  finite  state  MRP's)  is  Interesting  to  think 
about  since  MRP's  with  vastly  different  state  spaces  can  be  "close"  in 
the  metric. 

Perhaps  the  most  fundamental  observation  is  that  the  space  of 
MRP's  is  a  category.  There  are  several  morphlsms  between  MRP's  to  choose 
from.  Strong  lumpablllty,  weak  lumpability,  collapslbility  and  equiva¬ 
lence  are  all  morphisms  (theorem  2.6.6).  Consider  th<*  finite  and  count¬ 
able  state  MRP's.  Let  £  be  a  map  from  the  category  of  MRP's  to  the 
category  of  matrix  rings  defined  by 

£{(Q,y>}  -  Q. 

where  Q.  is  the  matrix  ring  generated  by  (Q(t)},  t  e  [0,«°].  Equivalence 
between  two  MRP's  is  too  weak  to  induce  a  functor  since  it  is  possible 
for  (Q,y)  ~  (Y, 6)  without  a  homomorphism  between  Q  and  V.  Theorems  2.5.1 
and  2.6.5  show  that  £  is  a  functor  if  the  morphism  used  is  strong 
lumpability,  weak  lumpability  or  collapslbility.  In  fact,  collapslbility 
occurs  between  two  MRP's  if  and  only  if  there  is  a  certain  homomorphism 
between  the  rings.  Thus,  from  this  point  of  view,  collapslbility  is  the 
most  pleasing  morphism  of  the  four.  Although  strong  and  weak  lumpability 
imply  homomorphisms,  the  converse  is  not  necessarily  true.  It  is  impor¬ 
tant  to  note  that  weak  lumpability,  collapslbility  and  equivalence  are 
identical  in  the  special  case  where  one  of  the  MRP's  is  a  renewal 
process.  It  has  still  not  been  formally  shown  that  collapslbility  and 
weak  lumpability  are  different,  although  in  the  finite  or  countable  case, 
weak  lumpability  clearly  implies  collapslbility,  and  in  view  of  (2.6.1) 
and  (2.6.2)  the  converse  is  doubtful. 
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Let  us  examine  these  morphisms  in  a  general  setting.  Let  (Q,y) 
be  a  MRP  with  state  space  (S^,  and  let  (Y,X)  be  a  MRP  with  state 

space  (Sj,  SP ^)  •  Let  <t>  be  a  measurable  map  of  onto  S2.  Say 
(Q,y)  Is  {Xn,  Tn>  and  (Y,X)  is  { Z n,  VQ}. 

Since  (Q,y)  is  a  MRP, 

p(Vi c  Vi  i  ‘lx„  ■  s>  ■  W'1 

is  a  regular  conditional  probability  on  «9^.  Assume  that  P(X^  e  a|  <X>)  (s) 
is  also  a  regular  conditional  probability  on  <9^.  Since 

Vs  e  <t_1(u),  P(XQ  e  A|<J))(s)  =  P(XQ  e  A|x0  e  $>_1(u» 

we  can  denote  P(Xq  e  A |  <X> )  (s)  by  P(u,A)  where  u  ■  <3>(s) .  P(u,*)  is  a 

measure  on  <9^  for  each  u  e  «9^,  and  P(‘,A)  is  e9^-measurable  for  every 
A  e  <9J.  P(u,A)  is  analogous  to  II  in  section  2.6. 

Let  £2  be  the  set  of  all  maps  Q:  x  <9^  *"  ®  such  that  for 

each  s  e  S^,  Q(s,*)  is  a  (signed)  measure  on  ^  and  for  each  A  e  *9^, 
Q(*,A)  is  «9^-measurable.  Likewise  let  22  be  the  set  of  all  such  maps 
Y:  S2  x  «9^  Define  addition  in  Q,  and  22  to  be  pointwise  addition 

and  define  multiplication  to  be 

Q1Q2(s,A)  -  j\ 51(8,dx)Q2(x,A). 

Let  Q  C  ^  be  the  ring  generated  by  {Q (t) }  and  let  V  C  22 
be  the  ring  generated  by  {Y(t)>.  Define  the  map  E  :  Q.  V  to  be 

Z(Q)uB  =  /q(s,  <J>-1(B))P(u,ds). 

The  morphisms  can  now  be  defined  in  the  general  setting. 

Definition  5.4.1.  Q  is  strongly  lumpable  to  Y  via  $>  if  for 
every  probability  measure  y  on  (S^,  <9^)»  {  4>(Xn),  Tn)  is  a  MRP  on 
(Sj.  SP2)  with  kernel  Y(t). 
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Definition  5.4.2.  Q  is  weakly  lumpable  to  Y  via  $  if  for  some 
probability  measure  p  on  (Sj,  if^i,  {  <t>(Xn) ,  Tn)  is  a  MRP  on  (S2,  if  £ 
with  kernel  Y(t). 

Definition  5.4.3.  (Q,y)  is  collapsible  to  Y  via  <t>  if  Z  is 

a  homomorphism. 

Definition  5.4.4.  (Q,p)  is  equivalent  to  (Y,X)  if  {Tn}  and  {V^} 

have  the  same  distribution. 
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